[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:36 dist-test-slave-b5jz krb5kdc[2010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:36 dist-test-slave-b5jz krb5kdc[2010](info): set up 2 sockets
Sep 23 22:07:36 dist-test-slave-b5jz krb5kdc[2010](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260923 22:07:37.480516 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:07:37 dist-test-slave-b5jz krb5kdc[2010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:07:37.594432 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33385
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:33385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:37.672969 2023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:37.673206 2023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:37.673244 2023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:37.674396 2023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:37.674467 2023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:37.674490 2023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:37.674510 2023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:37.674543 2023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:37.676019 2023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:33385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:37.676321 2023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:37.676548 2023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:37.681851 2028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:37.681957 2029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:37.681839 2031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:37.682046 2023 server_base.cc:1034] running on GCE node
I20260923 22:07:37.682341 2023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:37.682623 2023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:37.683781 2023 hybrid_clock.cc:648] HybridClock initialized: now 1790201257683757 us; error 59 us; skew 500 ppm
Sep 23 22:07:37 dist-test-slave-b5jz krb5kdc[2010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201257, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:37.691323 2023 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:07:37.691797 2023 webserver.cc:460] Webserver started at http://127.1.244.254:44465/ using document root <none> and password file <none>
I20260923 22:07:37.692018 2023 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:37.692083 2023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:37.692227 2023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:37.693243 2023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "059c1856dc014a0780675fc37f33cba9"
format_stamp: "Formatted at 2026-09-23 22:07:37 on dist-test-slave-b5jz"
server_key: "74816f2aa9e557298508b6364a1332b9"
server_key_iv: "c31fbbe28cfd980a466daf509461d489"
server_key_version: "encryptionkey@0"
I20260923 22:07:37.693612 2023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "059c1856dc014a0780675fc37f33cba9"
format_stamp: "Formatted at 2026-09-23 22:07:37 on dist-test-slave-b5jz"
server_key: "74816f2aa9e557298508b6364a1332b9"
server_key_iv: "c31fbbe28cfd980a466daf509461d489"
server_key_version: "encryptionkey@0"
I20260923 22:07:37.694921 2023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:07:37.695674 2038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:37.695845 2023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:37.695976 2023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "059c1856dc014a0780675fc37f33cba9"
format_stamp: "Formatted at 2026-09-23 22:07:37 on dist-test-slave-b5jz"
server_key: "74816f2aa9e557298508b6364a1332b9"
server_key_iv: "c31fbbe28cfd980a466daf509461d489"
server_key_version: "encryptionkey@0"
I20260923 22:07:37.696048 2023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:37.705206 2023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:37.706960 2023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:37.707125 2023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:37.711127 2023 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:33385
I20260923 22:07:37.711160 2090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:33385 every 8 connection(s)
I20260923 22:07:37.711519 2023 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:37.712344 2091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:37.714399 2091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9: Bootstrap starting.
I20260923 22:07:37.714901 2091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:37.715124 2091 log.cc:824] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:37.715866 2091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9: No bootstrap required, opened a new log
I20260923 22:07:37.717403 2091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059c1856dc014a0780675fc37f33cba9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33385 } }
I20260923 22:07:37.717535 2091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:37.717557 2091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059c1856dc014a0780675fc37f33cba9, State: Initialized, Role: FOLLOWER
I20260923 22:07:37.717686 2091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059c1856dc014a0780675fc37f33cba9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33385 } }
I20260923 22:07:37.717761 2091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:37.717783 2091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:37.717813 2091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:37.718333 2091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059c1856dc014a0780675fc37f33cba9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33385 } }
I20260923 22:07:37.718415 2091 leader_election.cc:302] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 059c1856dc014a0780675fc37f33cba9; no voters:
I20260923 22:07:37.718549 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 2023
I20260923 22:07:37.718647 2091 leader_election.cc:288] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:37.718659 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:07:37.718664 2094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:37.718782 2094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [term 1 LEADER]: Becoming Leader. State: Replica: 059c1856dc014a0780675fc37f33cba9, State: Running, Role: LEADER
I20260923 22:07:37.718875 2094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059c1856dc014a0780675fc37f33cba9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33385 } }
I20260923 22:07:37.718904 2091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:37.719189 2096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 059c1856dc014a0780675fc37f33cba9. Latest consensus state: current_term: 1 leader_uuid: "059c1856dc014a0780675fc37f33cba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059c1856dc014a0780675fc37f33cba9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33385 } } }
I20260923 22:07:37.719422 2003 external_mini_cluster.cc:1428] Setting key 5eab450083cf7d03af229c1c60391893
I20260923 22:07:37.719415 2096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:37.719347 2095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "059c1856dc014a0780675fc37f33cba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059c1856dc014a0780675fc37f33cba9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33385 } } }
I20260923 22:07:37.719586 2095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:37.719980 2098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:37.720481 2098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:37.721833 2098 catalog_manager.cc:1347] Generated new cluster ID: 089074b83b464d52a709645bc97aa205
I20260923 22:07:37.721882 2098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:07:37.729512 2098 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:37.730156 2098 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:37.732303 2098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 059c1856dc014a0780675fc37f33cba9: Generated new TSK 0
I20260923 22:07:37.732470 2098 catalog_manager.cc:1514] Initializing in-progress tserver states...
Sep 23 22:07:37 dist-test-slave-b5jz krb5kdc[2010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:37.737273 2105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:37.720325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56002 (local address 127.1.244.254:33385)
0923 22:07:37.720492 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:37.720501 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:37.721603 (+ 1102us) server_negotiation.cc:407] Connection header received
0923 22:07:37.723959 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:37.723966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:37.724024 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:37.724149 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:37.726162 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:37.727524 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:37.729268 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:37.729850 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:37.729959 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:37.733862 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:37.733866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:37.733866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:37.733877 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:37.735488 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:37.735944 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:37.735949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:37.735950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:37.736033 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:37.736292 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:37.736293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:37.736293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:37.736498 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:37.736633 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:37.736919 (+ 286us) server_negotiation.cc:299] Negotiation successful
0923 22:07:37.737056 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:07:37.764329 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33385
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:37.855963 2117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:37.856155 2117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:37.856179 2117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:37.857340 2117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:37.857383 2117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:37.857410 2117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:07:37.858963 2117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33385
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:37.859195 2117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:37.859445 2117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:37.865353 2123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:37.865373 2122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:37.865680 2117 server_base.cc:1034] running on GCE node
W20260923 22:07:37.865430 2125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:37.865935 2117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:37.866200 2117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:37.867365 2117 hybrid_clock.cc:648] HybridClock initialized: now 1790201257867355 us; error 31 us; skew 500 ppm
Sep 23 22:07:37 dist-test-slave-b5jz krb5kdc[2010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201257, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:37.870136 2117 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:07:37.870549 2117 webserver.cc:460] Webserver started at http://127.1.244.193:36119/ using document root <none> and password file <none>
I20260923 22:07:37.870774 2117 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:37.870841 2117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:37.871001 2117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:37.872108 2117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "769e204981b54184b907e59c4d249e7c"
format_stamp: "Formatted at 2026-09-23 22:07:37 on dist-test-slave-b5jz"
server_key: "c47f9dea99e7ec52d0e139bd51450dab"
server_key_iv: "f0dead6546c5ea85c643d0c1b7edc5d7"
server_key_version: "encryptionkey@0"
I20260923 22:07:37.872467 2117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "769e204981b54184b907e59c4d249e7c"
format_stamp: "Formatted at 2026-09-23 22:07:37 on dist-test-slave-b5jz"
server_key: "c47f9dea99e7ec52d0e139bd51450dab"
server_key_iv: "f0dead6546c5ea85c643d0c1b7edc5d7"
server_key_version: "encryptionkey@0"
I20260923 22:07:37.873835 2117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 22:07:37.874595 2132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:37.874756 2117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:37.874861 2117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "769e204981b54184b907e59c4d249e7c"
format_stamp: "Formatted at 2026-09-23 22:07:37 on dist-test-slave-b5jz"
server_key: "c47f9dea99e7ec52d0e139bd51450dab"
server_key_iv: "f0dead6546c5ea85c643d0c1b7edc5d7"
server_key_version: "encryptionkey@0"
I20260923 22:07:37.874934 2117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:37.887481 2117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:37.889389 2117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:37.889544 2117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:37.889746 2117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:37.890038 2117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:37.890091 2117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:37.890130 2117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:37.890158 2117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:37.896130 2117 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:40849
I20260923 22:07:37.896175 2245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:40849 every 8 connection(s)
I20260923 22:07:37.896453 2117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:07:37.899255 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 2117
I20260923 22:07:37.899339 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:37.899518 2003 external_mini_cluster.cc:1428] Setting key ee55b7c0b3cdc678facb13977b6f2781
Sep 23 22:07:37 dist-test-slave-b5jz krb5kdc[2010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201257, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:37.906368 2105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:37.897094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:36783 (local address 127.1.244.254:33385)
0923 22:07:37.897169 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:37.897171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:37.897309 (+ 138us) server_negotiation.cc:407] Connection header received
0923 22:07:37.897760 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:37.897761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:37.897770 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:37.897817 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:37.898769 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:37.899441 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:37.900698 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:37.901040 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:37.901114 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:37.903479 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:37.903480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:37.903481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:37.903488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:37.904919 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:37.905429 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:37.905430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:37.905430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:37.905488 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:37.905776 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:37.905776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:37.905776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:37.905869 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:37.905917 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:37.906241 (+ 324us) server_negotiation.cc:299] Negotiation successful
0923 22:07:37.906305 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:07:37.906613 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:37.897055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33385 (local address 127.1.244.193:36783)
0923 22:07:37.897201 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:37.897234 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:37.897685 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:37.897883 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:37.897886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:37.898460 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:37.898688 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:37.898694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:37.899525 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:37.899526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:37.900616 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:37.900618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:37.901122 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:37.901122 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:37.901216 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:37.901714 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:37.901730 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:37.903393 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:37.905003 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:37.905010 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:37.905013 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:37.905351 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:37.905558 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:37.905559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:37.905559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:37.905708 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:37.905934 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:37.905940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:37.906095 (+ 155us) client_negotiation.cc:769] Sending connection context
0923 22:07:37.906223 (+ 128us) client_negotiation.cc:240] Negotiation successful
0923 22:07:37.906382 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":88,"threads_started":1}
I20260923 22:07:37.907158 2246 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33385
I20260923 22:07:37.907255 2246 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:37.907444 2246 heartbeater.cc:511] Master 127.1.244.254:33385 requested a full tablet report, sending...
I20260923 22:07:37.907938 2055 ts_manager.cc:194] Registered new tserver with Master: 769e204981b54184b907e59c4d249e7c (127.1.244.193:40849)
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260923 22:07:37.908600 2055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:36783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:07:37.926434 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33385
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:38.009318 2251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:38.009536 2251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:38.009559 2251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:38.010691 2251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:38.010733 2251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:38.010761 2251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:07:38.012262 2251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33385
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:38.012512 2251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:38.012734 2251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:38.017490 2257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:38.017513 2256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:38.017554 2259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:38.017887 2251 server_base.cc:1034] running on GCE node
I20260923 22:07:38.018119 2251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:38.018349 2251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:38.019490 2251 hybrid_clock.cc:648] HybridClock initialized: now 1790201258019451 us; error 54 us; skew 500 ppm
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201258, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:38.022504 2251 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:07:38.022933 2251 webserver.cc:460] Webserver started at http://127.1.244.194:42065/ using document root <none> and password file <none>
I20260923 22:07:38.023175 2251 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:38.023241 2251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:38.023391 2251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:38.024451 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "9cb325198d5d4c018fe9bb2305bfe1f5"
format_stamp: "Formatted at 2026-09-23 22:07:38 on dist-test-slave-b5jz"
server_key: "5c723dc31f55c1350c13930291efc8cc"
server_key_iv: "52eeab782c298bf6bc50d9a5c7ebb7a3"
server_key_version: "encryptionkey@0"
I20260923 22:07:38.024819 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "9cb325198d5d4c018fe9bb2305bfe1f5"
format_stamp: "Formatted at 2026-09-23 22:07:38 on dist-test-slave-b5jz"
server_key: "5c723dc31f55c1350c13930291efc8cc"
server_key_iv: "52eeab782c298bf6bc50d9a5c7ebb7a3"
server_key_version: "encryptionkey@0"
I20260923 22:07:38.026082 2251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:38.026775 2266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:38.026947 2251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:38.027053 2251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "9cb325198d5d4c018fe9bb2305bfe1f5"
format_stamp: "Formatted at 2026-09-23 22:07:38 on dist-test-slave-b5jz"
server_key: "5c723dc31f55c1350c13930291efc8cc"
server_key_iv: "52eeab782c298bf6bc50d9a5c7ebb7a3"
server_key_version: "encryptionkey@0"
I20260923 22:07:38.027127 2251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:38.035182 2251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:38.037181 2251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:38.037343 2251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:38.037555 2251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:38.037837 2251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:38.037891 2251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:38.037930 2251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:38.037957 2251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:38.044440 2251 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:42327
I20260923 22:07:38.044477 2379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:42327 every 8 connection(s)
I20260923 22:07:38.044792 2251 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201258, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:38.052340 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 2251
I20260923 22:07:38.052448 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:38.052631 2003 external_mini_cluster.cc:1428] Setting key 765817e9357feb1f2639b928bbc5e2e6
I20260923 22:07:38.054571 2105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:38.045473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:54271 (local address 127.1.244.254:33385)
0923 22:07:38.045516 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:38.045519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:38.046099 (+ 580us) server_negotiation.cc:407] Connection header received
0923 22:07:38.046630 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:38.046631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:38.046641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:38.046708 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:38.047637 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.048141 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.049207 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.049383 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.049443 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:38.051512 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:38.051513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:38.051513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:38.051520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:38.053028 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:38.053485 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:38.053486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:38.053486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:38.053550 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:38.053898 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:38.053899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:38.053899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:38.054045 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:38.054116 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:38.054347 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 22:07:38.054439 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:07:38.054633 2382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:38.045441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33385 (local address 127.1.244.194:54271)
0923 22:07:38.045584 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:38.045622 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:38.046513 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:38.046761 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:38.046764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:38.047319 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:38.047497 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:38.047501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.048215 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.048216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:38.049126 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:38.049128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.049462 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.049463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:38.049556 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:38.050033 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:38.050041 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:38.051429 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:38.053099 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:38.053105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:38.053107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:38.053401 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:38.053709 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:38.053710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:38.053710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:38.053815 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:38.054113 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:38.054117 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:38.054224 (+ 107us) client_negotiation.cc:769] Sending connection context
0923 22:07:38.054304 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 22:07:38.054418 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260923 22:07:38.055112 2380 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33385
I20260923 22:07:38.055223 2380 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:38.055404 2380 heartbeater.cc:511] Master 127.1.244.254:33385 requested a full tablet report, sending...
I20260923 22:07:38.055840 2055 ts_manager.cc:194] Registered new tserver with Master: 9cb325198d5d4c018fe9bb2305bfe1f5 (127.1.244.194:42327)
I20260923 22:07:38.056214 2055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:54271
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:07:38.077652 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33385
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:38.157159 2385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:38.157382 2385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:38.157423 2385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:38.158573 2385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:38.158649 2385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:38.158694 2385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:07:38.160619 2385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33385
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:38.160877 2385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:38.161101 2385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:38.165863 2391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:38.165871 2393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:38.166085 2385 server_base.cc:1034] running on GCE node
W20260923 22:07:38.165871 2390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:38.166342 2385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:38.166575 2385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:38.167732 2385 hybrid_clock.cc:648] HybridClock initialized: now 1790201258167702 us; error 40 us; skew 500 ppm
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201258, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:38.170115 2385 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:07:38.170584 2385 webserver.cc:460] Webserver started at http://127.1.244.195:36445/ using document root <none> and password file <none>
I20260923 22:07:38.170807 2385 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:38.170874 2385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:38.171026 2385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:38.172102 2385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "45c91891da5c405c94048bf3bd9c9dd8"
format_stamp: "Formatted at 2026-09-23 22:07:38 on dist-test-slave-b5jz"
server_key: "a20b68fa4785d57a85a2736345f46301"
server_key_iv: "ed7f5f307332fe2cb95327033f159ef7"
server_key_version: "encryptionkey@0"
I20260923 22:07:38.172465 2385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "45c91891da5c405c94048bf3bd9c9dd8"
format_stamp: "Formatted at 2026-09-23 22:07:38 on dist-test-slave-b5jz"
server_key: "a20b68fa4785d57a85a2736345f46301"
server_key_iv: "ed7f5f307332fe2cb95327033f159ef7"
server_key_version: "encryptionkey@0"
I20260923 22:07:38.173729 2385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:38.174419 2400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:38.174590 2385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:38.174696 2385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "45c91891da5c405c94048bf3bd9c9dd8"
format_stamp: "Formatted at 2026-09-23 22:07:38 on dist-test-slave-b5jz"
server_key: "a20b68fa4785d57a85a2736345f46301"
server_key_iv: "ed7f5f307332fe2cb95327033f159ef7"
server_key_version: "encryptionkey@0"
I20260923 22:07:38.174768 2385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:38.189684 2385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:38.191428 2385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:38.191675 2385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:38.191903 2385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:38.192217 2385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:38.192268 2385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:38.192309 2385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:38.192353 2385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:38.198966 2385 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:41297
I20260923 22:07:38.199031 2513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:41297 every 8 connection(s)
I20260923 22:07:38.199353 2385 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:07:38.202169 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 2385
I20260923 22:07:38.202267 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:38.202457 2003 external_mini_cluster.cc:1428] Setting key 882142d06dafff50af8859496fde492b
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201258, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:38.208531 2105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:38.200146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:45151 (local address 127.1.244.254:33385)
0923 22:07:38.200204 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:38.200206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:38.200412 (+ 206us) server_negotiation.cc:407] Connection header received
0923 22:07:38.200842 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:38.200843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:38.200851 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:38.200904 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:38.201731 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.202206 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.203397 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.203691 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.203763 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:38.205707 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:38.205708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:38.205709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:38.205715 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:38.207091 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:38.207523 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:38.207524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:38.207525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:38.207610 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:38.207874 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:38.207875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:38.207875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:38.207996 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:38.208086 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:38.208364 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 22:07:38.208443 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:07:38.208716 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:38.200131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33385 (local address 127.1.244.195:45151)
0923 22:07:38.200295 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:38.200333 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:38.200765 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:38.200955 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:38.200958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:38.201486 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:38.201659 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:38.201663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.202320 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.202321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:38.203309 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:38.203312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.203773 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.203773 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:38.203873 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:38.204287 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:38.204295 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:38.205603 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:38.207191 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:38.207197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:38.207199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:38.207415 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:38.207683 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:38.207684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:38.207685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:38.207796 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:38.208077 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:38.208083 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:38.208218 (+ 135us) client_negotiation.cc:769] Sending connection context
0923 22:07:38.208338 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 22:07:38.208488 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260923 22:07:38.209177 2514 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33385
I20260923 22:07:38.209270 2514 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:38.209427 2514 heartbeater.cc:511] Master 127.1.244.254:33385 requested a full tablet report, sending...
I20260923 22:07:38.209826 2055 ts_manager.cc:194] Registered new tserver with Master: 45c91891da5c405c94048bf3bd9c9dd8 (127.1.244.195:41297)
I20260923 22:07:38.210191 2055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:45151
I20260923 22:07:38.214393 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201258, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201258, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:07:38.237020 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:38.229239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.1.244.193:40849)
0923 22:07:38.229439 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:38.229445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:38.229455 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:07:38.229489 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:38.229491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:38.229507 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:38.229588 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:38.230202 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.230919 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.231777 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.232106 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.232170 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:38.234403 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:38.234410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:38.234410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:38.234416 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:38.235864 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:38.236240 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:38.236243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:38.236244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:38.236321 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:38.236525 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:38.236527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:38.236527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:38.236667 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:38.236778 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:38.236849 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:07:38.236941 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
W20260923 22:07:38.237365 2160 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58292
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:07:38.259091 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:38.251949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58298 (local address 127.1.244.193:40849)
0923 22:07:38.252012 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:38.252013 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:38.252472 (+ 459us) server_negotiation.cc:407] Connection header received
0923 22:07:38.252565 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:38.252566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:38.252580 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:38.252653 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:38.253190 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.253675 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.254439 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:38.254645 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:38.254732 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:38.256831 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:38.256833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:38.256833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:38.256841 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:38.257980 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:38.258347 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:38.258348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:38.258349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:38.258409 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:38.258614 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:38.258615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:38.258616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:38.258711 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:38.258833 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:38.258911 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 22:07:38.258992 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:07:38.259588 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 2117
I20260923 22:07:38.264168 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 2251
I20260923 22:07:38.269554 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 2385
I20260923 22:07:38.273480 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 2023
2026-09-23T22:07:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1833 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2528](info): set up 2 sockets
Sep 23 22:07:38 dist-test-slave-b5jz krb5kdc[2528](info): commencing operation
krb5kdc: starting...
W20260923 22:07:39.299137 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:07:39 dist-test-slave-b5jz krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:07:39.410313 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37513
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:37513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:39.488674 2541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:39.488895 2541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:39.488933 2541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:39.490561 2541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:39.490617 2541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:39.490638 2541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:39.490658 2541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:39.490685 2541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:39.492197 2541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:37513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:39.492384 2541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:39.492584 2541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:39.497300 2546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:39.497349 2547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:39.497328 2549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:39.497462 2541 server_base.cc:1034] running on GCE node
I20260923 22:07:39.497718 2541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:39.497996 2541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:39.499157 2541 hybrid_clock.cc:648] HybridClock initialized: now 1790201259499138 us; error 38 us; skew 500 ppm
Sep 23 22:07:39 dist-test-slave-b5jz krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201259, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:39.501847 2541 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:07:39.502298 2541 webserver.cc:460] Webserver started at http://127.1.244.254:40125/ using document root <none> and password file <none>
I20260923 22:07:39.502501 2541 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:39.502542 2541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:39.502628 2541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:39.503662 2541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "65bd63d163db4b7db6568b20aeb8c6d5"
format_stamp: "Formatted at 2026-09-23 22:07:39 on dist-test-slave-b5jz"
server_key: "91cd063f1cb513dcd96d998c822b534c"
server_key_iv: "75c6d9f4ab8b7b84ef2685c73ae86133"
server_key_version: "encryptionkey@0"
I20260923 22:07:39.504006 2541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "65bd63d163db4b7db6568b20aeb8c6d5"
format_stamp: "Formatted at 2026-09-23 22:07:39 on dist-test-slave-b5jz"
server_key: "91cd063f1cb513dcd96d998c822b534c"
server_key_iv: "75c6d9f4ab8b7b84ef2685c73ae86133"
server_key_version: "encryptionkey@0"
I20260923 22:07:39.505249 2541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:07:39.505985 2556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:39.506182 2541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:07:39.506273 2541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "65bd63d163db4b7db6568b20aeb8c6d5"
format_stamp: "Formatted at 2026-09-23 22:07:39 on dist-test-slave-b5jz"
server_key: "91cd063f1cb513dcd96d998c822b534c"
server_key_iv: "75c6d9f4ab8b7b84ef2685c73ae86133"
server_key_version: "encryptionkey@0"
I20260923 22:07:39.506346 2541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:39.531358 2541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:39.533658 2541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:39.533825 2541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:39.537690 2541 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:37513
I20260923 22:07:39.537777 2608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:37513 every 8 connection(s)
I20260923 22:07:39.538084 2541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:39.538908 2609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:39.541172 2609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5: Bootstrap starting.
I20260923 22:07:39.541746 2609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:39.542034 2609 log.cc:824] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:39.542709 2609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5: No bootstrap required, opened a new log
I20260923 22:07:39.544344 2609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bd63d163db4b7db6568b20aeb8c6d5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37513 } }
I20260923 22:07:39.544464 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 2541
I20260923 22:07:39.544503 2609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:39.544531 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:07:39.544553 2609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65bd63d163db4b7db6568b20aeb8c6d5, State: Initialized, Role: FOLLOWER
I20260923 22:07:39.544716 2003 external_mini_cluster.cc:1428] Setting key bbe72c15369f39f6f347b3a6a8017966
I20260923 22:07:39.544713 2609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bd63d163db4b7db6568b20aeb8c6d5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37513 } }
I20260923 22:07:39.544797 2609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:39.544840 2609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:39.544902 2609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:39.546008 2609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bd63d163db4b7db6568b20aeb8c6d5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37513 } }
I20260923 22:07:39.546131 2609 leader_election.cc:302] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65bd63d163db4b7db6568b20aeb8c6d5; no voters:
I20260923 22:07:39.546293 2609 leader_election.cc:288] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:39.546324 2614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:39.546480 2614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [term 1 LEADER]: Becoming Leader. State: Replica: 65bd63d163db4b7db6568b20aeb8c6d5, State: Running, Role: LEADER
I20260923 22:07:39.546545 2609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:39.546567 2614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bd63d163db4b7db6568b20aeb8c6d5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37513 } }
I20260923 22:07:39.546909 2614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65bd63d163db4b7db6568b20aeb8c6d5. Latest consensus state: current_term: 1 leader_uuid: "65bd63d163db4b7db6568b20aeb8c6d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bd63d163db4b7db6568b20aeb8c6d5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37513 } } }
I20260923 22:07:39.547011 2614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:39.546919 2615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65bd63d163db4b7db6568b20aeb8c6d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bd63d163db4b7db6568b20aeb8c6d5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37513 } } }
I20260923 22:07:39.547158 2615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:39.547279 2621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:39.547794 2621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:07:39 dist-test-slave-b5jz krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:39.552979 2621 catalog_manager.cc:1347] Generated new cluster ID: 32adb2e256a44b7ebc30fe2e4f91a4b8
I20260923 22:07:39.553038 2621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:07:39.555823 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:39.545226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.1.244.254:37513)
0923 22:07:39.545411 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:39.545420 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:39.545456 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:07:39.545562 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:39.545567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:39.545606 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:39.545715 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:39.546518 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.548007 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.549046 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.549640 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.549767 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:39.552222 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:39.552226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:39.552227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:39.552238 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:39.554174 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:39.554638 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:39.554641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:39.554641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:39.554753 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:39.554967 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:39.554968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:39.554968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:39.555241 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:39.555386 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:39.555499 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 22:07:39.555643 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
I20260923 22:07:39.567090 2621 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:39.567670 2621 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:39.574720 2621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65bd63d163db4b7db6568b20aeb8c6d5: Generated new TSK 0
I20260923 22:07:39.574887 2621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:07:39.607870 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37513
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:39.687108 2635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:39.687337 2635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:39.687374 2635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:39.688603 2635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:39.688663 2635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:39.688707 2635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:07:39.690181 2635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37513
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:39.690438 2635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:39.690656 2635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:39.695370 2643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:39.695346 2640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:39.695359 2641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:39.695708 2635 server_base.cc:1034] running on GCE node
I20260923 22:07:39.695956 2635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:39.696192 2635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:39.697353 2635 hybrid_clock.cc:648] HybridClock initialized: now 1790201259697348 us; error 41 us; skew 500 ppm
Sep 23 22:07:39 dist-test-slave-b5jz krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201259, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:39.699867 2635 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:07:39.700289 2635 webserver.cc:460] Webserver started at http://127.1.244.193:46539/ using document root <none> and password file <none>
I20260923 22:07:39.700474 2635 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:39.700515 2635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:39.700624 2635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:39.701578 2635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "f6cfc95e12db45298f9d7ccb7a8225a3"
format_stamp: "Formatted at 2026-09-23 22:07:39 on dist-test-slave-b5jz"
server_key: "5cef7058d2cee378eb44052803ef7c9d"
server_key_iv: "2fbe9e005c3325a15f1af9a274c50d5d"
server_key_version: "encryptionkey@0"
I20260923 22:07:39.701885 2635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "f6cfc95e12db45298f9d7ccb7a8225a3"
format_stamp: "Formatted at 2026-09-23 22:07:39 on dist-test-slave-b5jz"
server_key: "5cef7058d2cee378eb44052803ef7c9d"
server_key_iv: "2fbe9e005c3325a15f1af9a274c50d5d"
server_key_version: "encryptionkey@0"
I20260923 22:07:39.703092 2635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:39.703800 2650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:39.703990 2635 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:39.704061 2635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "f6cfc95e12db45298f9d7ccb7a8225a3"
format_stamp: "Formatted at 2026-09-23 22:07:39 on dist-test-slave-b5jz"
server_key: "5cef7058d2cee378eb44052803ef7c9d"
server_key_iv: "2fbe9e005c3325a15f1af9a274c50d5d"
server_key_version: "encryptionkey@0"
I20260923 22:07:39.704110 2635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:39.716010 2635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:39.717674 2635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:39.717775 2635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:39.717934 2635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:39.718155 2635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:39.718184 2635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:39.718204 2635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:39.718216 2635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:39.723764 2635 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:37421
I20260923 22:07:39.723819 2763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:37421 every 8 connection(s)
I20260923 22:07:39.724100 2635 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:07:39 dist-test-slave-b5jz krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201259, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:39.732425 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:39.724787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:37093 (local address 127.1.244.254:37513)
0923 22:07:39.724838 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:39.724840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:39.725041 (+ 201us) server_negotiation.cc:407] Connection header received
0923 22:07:39.725895 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:39.725896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:39.725906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:39.725966 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:39.726508 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.726990 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.727941 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.728135 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.728189 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:39.730203 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:39.730204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:39.730204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:39.730213 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:39.731370 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:39.731735 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:39.731736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:39.731736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:39.731787 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:39.731976 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:39.731976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:39.731977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:39.732064 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:39.732110 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:39.732305 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 22:07:39.732363 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:07:39.732547 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 2635
I20260923 22:07:39.732622 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:39.732692 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:39.724771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37513 (local address 127.1.244.193:37093)
0923 22:07:39.724909 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:39.724941 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:39.725809 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:39.726024 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:39.726027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:39.726264 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:39.726440 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:39.726443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.727066 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.727067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:39.727874 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:39.727876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.728209 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.728210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:39.728310 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:39.728814 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:39.728821 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:39.730122 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:39.731439 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:39.731445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:39.731446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:39.731674 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:39.731845 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:39.731845 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:39.731845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:39.731936 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:39.732112 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:39.732116 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:39.732216 (+ 100us) client_negotiation.cc:769] Sending connection context
0923 22:07:39.732294 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:07:39.732455 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
I20260923 22:07:39.732792 2003 external_mini_cluster.cc:1428] Setting key 76c55a72f8e4c952c16e2f0229c556b7
I20260923 22:07:39.733234 2764 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37513
I20260923 22:07:39.733319 2764 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:39.733500 2764 heartbeater.cc:511] Master 127.1.244.254:37513 requested a full tablet report, sending...
I20260923 22:07:39.734061 2572 ts_manager.cc:194] Registered new tserver with Master: f6cfc95e12db45298f9d7ccb7a8225a3 (127.1.244.193:37421)
I20260923 22:07:39.734964 2572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:37093
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:07:39.758080 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37513
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:39.838060 2769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:39.838258 2769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:39.838284 2769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:39.839432 2769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:39.839473 2769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:39.839499 2769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:07:39.841014 2769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37513
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:39.841228 2769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:39.841449 2769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:39.846493 2774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:39.846480 2777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:39.846467 2775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:39.846561 2769 server_base.cc:1034] running on GCE node
I20260923 22:07:39.846931 2769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:39.847199 2769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:39.848357 2769 hybrid_clock.cc:648] HybridClock initialized: now 1790201259848343 us; error 33 us; skew 500 ppm
Sep 23 22:07:39 dist-test-slave-b5jz krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201259, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:39.850796 2769 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:07:39.851274 2769 webserver.cc:460] Webserver started at http://127.1.244.194:41701/ using document root <none> and password file <none>
I20260923 22:07:39.851516 2769 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:39.851625 2769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:39.851787 2769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:39.852837 2769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "725ad8f78d0e42e8916ed90daadb00dd"
format_stamp: "Formatted at 2026-09-23 22:07:39 on dist-test-slave-b5jz"
server_key: "ad96c1185992a293e3e69e17fc3784d6"
server_key_iv: "4fcabe63ecf78432f9272b5499ba1319"
server_key_version: "encryptionkey@0"
I20260923 22:07:39.853256 2769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "725ad8f78d0e42e8916ed90daadb00dd"
format_stamp: "Formatted at 2026-09-23 22:07:39 on dist-test-slave-b5jz"
server_key: "ad96c1185992a293e3e69e17fc3784d6"
server_key_iv: "4fcabe63ecf78432f9272b5499ba1319"
server_key_version: "encryptionkey@0"
I20260923 22:07:39.854550 2769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:07:39.855248 2784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:39.855407 2769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:39.855510 2769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "725ad8f78d0e42e8916ed90daadb00dd"
format_stamp: "Formatted at 2026-09-23 22:07:39 on dist-test-slave-b5jz"
server_key: "ad96c1185992a293e3e69e17fc3784d6"
server_key_iv: "4fcabe63ecf78432f9272b5499ba1319"
server_key_version: "encryptionkey@0"
I20260923 22:07:39.855618 2769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:39.873704 2769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:39.875685 2769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:39.875846 2769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:39.876091 2769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:39.876413 2769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:39.876470 2769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:39.876510 2769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:39.876554 2769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:39.882334 2769 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:42371
I20260923 22:07:39.882378 2897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:42371 every 8 connection(s)
I20260923 22:07:39.882699 2769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:07:39 dist-test-slave-b5jz krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201259, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:39.892433 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:39.883388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:41647 (local address 127.1.244.254:37513)
0923 22:07:39.883437 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:39.883439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:39.883647 (+ 208us) server_negotiation.cc:407] Connection header received
0923 22:07:39.884588 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:39.884589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:39.884599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:39.884657 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:39.885231 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.885718 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.886991 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.887378 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.887452 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:39.889579 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:39.889581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:39.889581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:39.889589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:39.890997 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:39.891418 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:39.891419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:39.891420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:39.891483 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:39.891774 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:39.891775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:39.891776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:39.891889 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:39.891969 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:39.892255 (+ 286us) server_negotiation.cc:299] Negotiation successful
0923 22:07:39.892343 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:07:39.892606 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:39.883354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37513 (local address 127.1.244.194:41647)
0923 22:07:39.883510 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:39.883576 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:39.884489 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:39.884711 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:39.884715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:39.884963 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:39.885160 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:39.885164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.885798 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.885798 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:39.886891 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:39.886893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:39.887473 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:39.887475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:39.887623 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:39.888142 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:39.888158 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:39.889476 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:39.891101 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:39.891108 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:39.891110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:39.891336 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:39.891576 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:39.891577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:39.891577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:39.891697 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:39.891968 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:39.891975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:39.892118 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 22:07:39.892223 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:07:39.892375 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260923 22:07:39.893043 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 2769
I20260923 22:07:39.893129 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:39.893146 2898 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37513
I20260923 22:07:39.893239 2898 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:39.893321 2003 external_mini_cluster.cc:1428] Setting key 87bceb3273b888b9c9ccb43dd61daefc
I20260923 22:07:39.893383 2898 heartbeater.cc:511] Master 127.1.244.254:37513 requested a full tablet report, sending...
I20260923 22:07:39.893731 2573 ts_manager.cc:194] Registered new tserver with Master: 725ad8f78d0e42e8916ed90daadb00dd (127.1.244.194:42371)
I20260923 22:07:39.894125 2573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:41647
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:07:39.916469 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37513
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:39.997179 2903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:39.997383 2903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:39.997407 2903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:39.998582 2903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:39.998626 2903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:39.998658 2903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:07:40.000237 2903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37513
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:40.000483 2903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:40.000700 2903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:40.005710 2909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:40.005824 2903 server_base.cc:1034] running on GCE node
W20260923 22:07:40.005726 2911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:40.005703 2908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:40.006124 2903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:40.006317 2903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:40.007447 2903 hybrid_clock.cc:648] HybridClock initialized: now 1790201260007442 us; error 35 us; skew 500 ppm
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201260, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:40.010098 2903 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:07:40.010569 2903 webserver.cc:460] Webserver started at http://127.1.244.195:45311/ using document root <none> and password file <none>
I20260923 22:07:40.010808 2903 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:40.010854 2903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:40.010984 2903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:40.012112 2903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "89f0c7206f634f6486327dc35a4bcaa5"
format_stamp: "Formatted at 2026-09-23 22:07:40 on dist-test-slave-b5jz"
server_key: "90f5a88311aa3027a72ad017b85e0140"
server_key_iv: "a5c27985979a811a7ef4f33ae0b67ecc"
server_key_version: "encryptionkey@0"
I20260923 22:07:40.012465 2903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "89f0c7206f634f6486327dc35a4bcaa5"
format_stamp: "Formatted at 2026-09-23 22:07:40 on dist-test-slave-b5jz"
server_key: "90f5a88311aa3027a72ad017b85e0140"
server_key_iv: "a5c27985979a811a7ef4f33ae0b67ecc"
server_key_version: "encryptionkey@0"
I20260923 22:07:40.013751 2903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:40.014463 2918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:40.014652 2903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:40.014720 2903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "89f0c7206f634f6486327dc35a4bcaa5"
format_stamp: "Formatted at 2026-09-23 22:07:40 on dist-test-slave-b5jz"
server_key: "90f5a88311aa3027a72ad017b85e0140"
server_key_iv: "a5c27985979a811a7ef4f33ae0b67ecc"
server_key_version: "encryptionkey@0"
I20260923 22:07:40.014822 2903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:40.031817 2903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:40.034159 2903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:40.034327 2903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:40.034564 2903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:40.034940 2903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:40.034994 2903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:40.035045 2903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:40.035073 2903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:40.040864 2903 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:42487
I20260923 22:07:40.040978 3031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:42487 every 8 connection(s)
I20260923 22:07:40.041282 2903 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201260, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:40.050697 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.042040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:35971 (local address 127.1.244.254:37513)
0923 22:07:40.042113 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.042116 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.042271 (+ 155us) server_negotiation.cc:407] Connection header received
0923 22:07:40.043220 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.043222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.043233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.043305 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:40.043912 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.044449 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.045426 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.045629 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.045692 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.048021 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:40.048023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:40.048024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:40.048032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:40.049450 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.049807 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.049808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.049808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.049862 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.050113 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.050114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.050114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.050221 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:40.050278 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.050543 (+ 265us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.050621 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 22:07:40.050741 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.041988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37513 (local address 127.1.244.195:35971)
0923 22:07:40.042141 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:40.042181 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:40.043110 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:40.043358 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:40.043361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:40.043622 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:40.043808 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.043812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.044535 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.044536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.045334 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.045336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.045706 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.045707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.045800 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.046337 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:40.046345 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:40.047917 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:40.049526 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:40.049531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:40.049533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:40.049739 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:40.049960 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:40.049960 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:40.049961 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:40.050057 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:40.050279 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:40.050283 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:40.050388 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 22:07:40.050468 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 22:07:40.050584 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260923 22:07:40.051043 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 2903
I20260923 22:07:40.051141 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:40.051255 3032 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37513
I20260923 22:07:40.051309 2003 external_mini_cluster.cc:1428] Setting key badf82a93b801a0d8d00fa3d92742b6a
I20260923 22:07:40.051369 3032 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:40.051625 3032 heartbeater.cc:511] Master 127.1.244.254:37513 requested a full tablet report, sending...
I20260923 22:07:40.052068 2572 ts_manager.cc:194] Registered new tserver with Master: 89f0c7206f634f6486327dc35a4bcaa5 (127.1.244.195:42487)
I20260923 22:07:40.052518 2572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:35971
I20260923 22:07:40.063091 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:07:40.070333 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.064644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.1.244.254:37513)
0923 22:07:40.064704 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.064706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.064765 (+ 59us) server_negotiation.cc:407] Connection header received
0923 22:07:40.064804 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.064805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.064813 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.064863 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:40.065374 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.065841 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.066500 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.066685 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.066737 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.067933 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:40.067934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:40.067934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:40.067940 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:40.069103 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.069413 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.069413 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.069414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.069465 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.069751 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.069752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.069752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.069921 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:40.070015 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.070167 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.070232 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:07:40.072746 2572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:07:40.074342 2572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:40.080785 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.076761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42371 (local address 127.0.0.1:33438)
0923 22:07:40.077024 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:40.077034 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:40.077076 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:40.077463 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:40.077464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:40.077466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:40.077526 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.077528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.078479 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.078480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.079530 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.079533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.080463 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.080464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.080552 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.080567 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:07:40.080649 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 22:07:40.080700 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":78,"threads_started":1}
I20260923 22:07:40.080792 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.076900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.1.244.194:42371)
0923 22:07:40.077310 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.077321 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.077329 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:07:40.077360 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.077362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.077373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.077462 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:40.077616 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.078373 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.079649 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.080381 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.080451 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.080509 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.080674 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.080714 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":82,"threads_started":1}
I20260923 22:07:40.081382 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.076955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42487 (local address 127.0.0.1:36722)
0923 22:07:40.077124 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:40.077134 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:40.077188 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:40.077538 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:40.077538 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:40.077540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:40.077573 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.077574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.079095 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.079096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.080189 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.080192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.081156 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.081157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.081219 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.081224 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:07:40.081275 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:07:40.081314 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":27,"thread_start_us":34,"threads_started":1}
I20260923 22:07:40.081513 3043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.076734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:37421 (local address 127.0.0.1:34596)
0923 22:07:40.076918 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:40.076962 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:40.077033 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:40.077413 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:40.077416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:40.077418 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:40.077490 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.077492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.079127 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.079128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.080173 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.080176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.081211 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.081212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.081291 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.081295 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:40.081366 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:07:40.081410 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":1,"thread_start_us":50,"threads_started":1}
I20260923 22:07:40.081606 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.077130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.1.244.195:42487)
0923 22:07:40.077310 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.077314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.077323 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:07:40.077357 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.077359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.077383 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.077453 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:40.077648 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.078958 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.080288 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.081004 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.081069 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.081119 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.081484 (+ 365us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.081533 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260923 22:07:40.081670 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.076864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.1.244.193:37421)
0923 22:07:40.077193 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.077197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.077204 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:07:40.077261 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.077264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.077281 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.077571 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:40.077645 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.078958 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.080277 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.081130 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.081208 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.081263 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.081381 (+ 118us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.081419 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":78,"threads_started":1}
I20260923 22:07:40.081645 2832 tablet_service.cc:1467] Processing CreateTablet for tablet ef65c52aeb2340599a31f76744524a26 (DEFAULT_TABLE table=test-table [id=dcac4050b5834e47882739f85b460662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:40.081987 2832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef65c52aeb2340599a31f76744524a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:40.082170 2966 tablet_service.cc:1467] Processing CreateTablet for tablet ef65c52aeb2340599a31f76744524a26 (DEFAULT_TABLE table=test-table [id=dcac4050b5834e47882739f85b460662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:40.082472 2966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef65c52aeb2340599a31f76744524a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:40.085410 3049 tablet_bootstrap.cc:492] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd: Bootstrap starting.
I20260923 22:07:40.085762 2698 tablet_service.cc:1467] Processing CreateTablet for tablet ef65c52aeb2340599a31f76744524a26 (DEFAULT_TABLE table=test-table [id=dcac4050b5834e47882739f85b460662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:40.086055 3049 tablet_bootstrap.cc:654] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:40.086063 2698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef65c52aeb2340599a31f76744524a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:40.086547 3049 log.cc:824] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:40.087251 3049 tablet_bootstrap.cc:492] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd: No bootstrap required, opened a new log
I20260923 22:07:40.087344 3049 ts_tablet_manager.cc:1397] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:40.087721 3050 tablet_bootstrap.cc:492] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5: Bootstrap starting.
I20260923 22:07:40.088343 3050 tablet_bootstrap.cc:654] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:40.088913 3050 log.cc:824] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:40.089186 3052 tablet_bootstrap.cc:492] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3: Bootstrap starting.
I20260923 22:07:40.089659 3050 tablet_bootstrap.cc:492] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5: No bootstrap required, opened a new log
I20260923 22:07:40.089711 3050 ts_tablet_manager.cc:1397] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:40.089915 3052 tablet_bootstrap.cc:654] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:40.089807 3049 raft_consensus.cc:348] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } }
I20260923 22:07:40.089996 3049 raft_consensus.cc:374] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:40.090045 3049 raft_consensus.cc:729] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725ad8f78d0e42e8916ed90daadb00dd, State: Initialized, Role: FOLLOWER
I20260923 22:07:40.090188 3049 consensus_queue.cc:260] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } }
I20260923 22:07:40.090456 3052 log.cc:824] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:40.090466 3049 ts_tablet_manager.cc:1428] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:07:40.090471 2898 heartbeater.cc:503] Master 127.1.244.254:37513 was elected leader, sending a full tablet report...
I20260923 22:07:40.091215 3052 tablet_bootstrap.cc:492] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3: No bootstrap required, opened a new log
I20260923 22:07:40.091284 3052 ts_tablet_manager.cc:1397] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:40.091714 3050 raft_consensus.cc:348] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } }
I20260923 22:07:40.091882 3050 raft_consensus.cc:374] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:40.091915 3050 raft_consensus.cc:729] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f0c7206f634f6486327dc35a4bcaa5, State: Initialized, Role: FOLLOWER
I20260923 22:07:40.092031 3050 consensus_queue.cc:260] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } }
I20260923 22:07:40.092275 3050 ts_tablet_manager.cc:1428] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:07:40.092315 3032 heartbeater.cc:503] Master 127.1.244.254:37513 was elected leader, sending a full tablet report...
I20260923 22:07:40.093273 3052 raft_consensus.cc:348] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } }
I20260923 22:07:40.093410 3052 raft_consensus.cc:374] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:40.093431 3052 raft_consensus.cc:729] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cfc95e12db45298f9d7ccb7a8225a3, State: Initialized, Role: FOLLOWER
I20260923 22:07:40.093499 3052 consensus_queue.cc:260] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } }
I20260923 22:07:40.093707 3052 ts_tablet_manager.cc:1428] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:07:40.093732 2764 heartbeater.cc:503] Master 127.1.244.254:37513 was elected leader, sending a full tablet report...
W20260923 22:07:40.133248 2899 tablet.cc:2369] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:07:40.197063 3057 raft_consensus.cc:482] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:40.197225 3057 raft_consensus.cc:504] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } }
I20260923 22:07:40.197674 3057 leader_election.cc:288] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 725ad8f78d0e42e8916ed90daadb00dd (127.1.244.194:42371), 89f0c7206f634f6486327dc35a4bcaa5 (127.1.244.195:42487)
I20260923 22:07:40.201911 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.197972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:54175 (local address 127.1.244.195:42487)
0923 22:07:40.198016 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.198019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.198185 (+ 166us) server_negotiation.cc:407] Connection header received
0923 22:07:40.198220 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.198220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.198228 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.198271 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:40.198485 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.199268 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.200893 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.201542 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.201629 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.201638 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.201802 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.201841 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:07:40.201916 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.197848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42371 (local address 127.1.244.193:52711)
0923 22:07:40.197908 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:40.197921 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:40.197969 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:40.198157 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:40.198158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:40.198160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:40.198237 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.198239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.199357 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.199357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.200776 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.200779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.201637 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.201638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.201717 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.201720 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:07:40.201778 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:07:40.201821 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":21}
I20260923 22:07:40.201951 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.197944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:52711 (local address 127.1.244.194:42371)
0923 22:07:40.197994 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.197997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.198009 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:07:40.198039 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.198039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.198052 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.198130 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:40.198346 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.199264 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.200921 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.201542 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.201630 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.201638 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.201802 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.201841 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:07:40.201922 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.197874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42487 (local address 127.1.244.193:54175)
0923 22:07:40.198072 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:40.198085 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:40.198165 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:40.198302 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:40.198303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:40.198304 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:40.198364 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.198365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.199357 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.199357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.200776 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:40.200779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.201638 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.201638 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:40.201716 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.201720 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:40.201778 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:07:40.201820 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":107,"thread_start_us":112,"threads_started":1}
I20260923 22:07:40.202371 2852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef65c52aeb2340599a31f76744524a26" candidate_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "725ad8f78d0e42e8916ed90daadb00dd" is_pre_election: true
I20260923 22:07:40.202569 2852 raft_consensus.cc:2454] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6cfc95e12db45298f9d7ccb7a8225a3 in term 0.
I20260923 22:07:40.202767 2986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef65c52aeb2340599a31f76744524a26" candidate_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f0c7206f634f6486327dc35a4bcaa5" is_pre_election: true
I20260923 22:07:40.202811 2651 leader_election.cc:302] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 725ad8f78d0e42e8916ed90daadb00dd, f6cfc95e12db45298f9d7ccb7a8225a3; no voters:
I20260923 22:07:40.202935 2986 raft_consensus.cc:2454] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6cfc95e12db45298f9d7ccb7a8225a3 in term 0.
I20260923 22:07:40.202945 3057 raft_consensus.cc:2781] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:40.202994 3057 raft_consensus.cc:482] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:40.203027 3057 raft_consensus.cc:3037] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:40.203816 3057 raft_consensus.cc:504] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } }
I20260923 22:07:40.203984 3057 leader_election.cc:288] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [CANDIDATE]: Term 1 election: Requested vote from peers 725ad8f78d0e42e8916ed90daadb00dd (127.1.244.194:42371), 89f0c7206f634f6486327dc35a4bcaa5 (127.1.244.195:42487)
I20260923 22:07:40.204138 2852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef65c52aeb2340599a31f76744524a26" candidate_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "725ad8f78d0e42e8916ed90daadb00dd"
I20260923 22:07:40.204169 2986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef65c52aeb2340599a31f76744524a26" candidate_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f0c7206f634f6486327dc35a4bcaa5"
I20260923 22:07:40.204252 2986 raft_consensus.cc:3037] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:40.204237 2852 raft_consensus.cc:3037] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:40.204921 2986 raft_consensus.cc:2454] T ef65c52aeb2340599a31f76744524a26 P 89f0c7206f634f6486327dc35a4bcaa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6cfc95e12db45298f9d7ccb7a8225a3 in term 1.
I20260923 22:07:40.205302 2652 leader_election.cc:302] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89f0c7206f634f6486327dc35a4bcaa5, f6cfc95e12db45298f9d7ccb7a8225a3; no voters:
I20260923 22:07:40.205435 3057 raft_consensus.cc:2781] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:40.205585 3057 raft_consensus.cc:686] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [term 1 LEADER]: Becoming Leader. State: Replica: f6cfc95e12db45298f9d7ccb7a8225a3, State: Running, Role: LEADER
I20260923 22:07:40.205581 2852 raft_consensus.cc:2454] T ef65c52aeb2340599a31f76744524a26 P 725ad8f78d0e42e8916ed90daadb00dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6cfc95e12db45298f9d7ccb7a8225a3 in term 1.
I20260923 22:07:40.205683 3057 consensus_queue.cc:237] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } }
I20260923 22:07:40.206691 2572 catalog_manager.cc:5495] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6cfc95e12db45298f9d7ccb7a8225a3 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6cfc95e12db45298f9d7ccb7a8225a3" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "725ad8f78d0e42e8916ed90daadb00dd" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f0c7206f634f6486327dc35a4bcaa5" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42487 } health_report { overall_health: UNKNOWN } } }
W20260923 22:07:40.224793 2765 tablet.cc:2369] T ef65c52aeb2340599a31f76744524a26 P f6cfc95e12db45298f9d7ccb7a8225a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:07:40.229645 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.221602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.1.244.193:37421)
0923 22:07:40.221677 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.221679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.221730 (+ 51us) server_negotiation.cc:407] Connection header received
0923 22:07:40.221782 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.221783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.221792 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.221837 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:40.222429 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.222874 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.223670 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.223912 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.223963 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.226870 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:40.226877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:40.226877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:40.226883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:40.228589 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.228912 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.228914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.228914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.228980 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.229138 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.229139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.229139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.229255 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:40.229366 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.229478 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.229541 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201260, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201260, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:07:40.253958 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.246257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.1.244.193:37421)
0923 22:07:40.246331 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.246333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.246446 (+ 113us) server_negotiation.cc:407] Connection header received
0923 22:07:40.246550 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.246551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.246562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.246641 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:40.247371 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.248116 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.248953 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.249203 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.249278 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.251509 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:40.251510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:40.251511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:40.251519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:40.252725 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.253101 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.253102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.253103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.253161 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.253379 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.253380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.253381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.253480 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:40.253582 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.253760 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.253861 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:07:40.275862 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:40.268952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34622 (local address 127.1.244.193:37421)
0923 22:07:40.269035 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:40.269038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:40.269130 (+ 92us) server_negotiation.cc:407] Connection header received
0923 22:07:40.269234 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:40.269235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:40.269243 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:40.269303 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:40.269828 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.270280 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.270995 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:40.271187 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:40.271259 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:40.273435 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:40.273436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:40.273437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:40.273444 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:40.274776 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.275120 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.275121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.275121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.275177 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:40.275368 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:40.275369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:40.275369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:40.275463 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:40.275582 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:40.275693 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 22:07:40.275766 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260923 22:07:40.276304 2678 tablet_service.cc:2908] Rejecting scan request for tablet ef65c52aeb2340599a31f76744524a26: Uninitialized: clean time has not yet been initialized
I20260923 22:07:40.277344 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 2635
I20260923 22:07:40.282375 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 2769
I20260923 22:07:40.286690 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 2903
I20260923 22:07:40.291260 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 2541
2026-09-23T22:07:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2019 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[3073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[3073](info): set up 2 sockets
Sep 23 22:07:40 dist-test-slave-b5jz krb5kdc[3073](info): commencing operation
krb5kdc: starting...
W20260923 22:07:41.319223 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:07:41 dist-test-slave-b5jz krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:07:41.430217 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46175
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:41.508544 3086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:41.508787 3086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:41.508823 3086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:41.509960 3086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:41.510027 3086 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:07:41.510052 3086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:41.510074 3086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:41.510106 3086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:41.510129 3086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:41.511617 3086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46175
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:41.511895 3086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:41.512113 3086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:41.516716 3094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:41.516711 3092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:41.516739 3091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:41.516911 3086 server_base.cc:1034] running on GCE node
I20260923 22:07:41.517280 3086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:41.517560 3086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:41.518733 3086 hybrid_clock.cc:648] HybridClock initialized: now 1790201261518717 us; error 45 us; skew 500 ppm
Sep 23 22:07:41 dist-test-slave-b5jz krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201261, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:41.521199 3086 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:07:41.521564 3086 webserver.cc:460] Webserver started at http://127.1.244.254:45967/ using document root <none> and password file <none>
I20260923 22:07:41.521731 3086 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:41.521768 3086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:41.521852 3086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:41.522825 3086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "b470ac46f76b4b93ad01b2c1603d9501"
format_stamp: "Formatted at 2026-09-23 22:07:41 on dist-test-slave-b5jz"
server_key: "fe4a545c0d5b6b02fefcd97211a9f029"
server_key_iv: "969d4a2a9fe702567d5752eaaedac3af"
server_key_version: "encryptionkey@0"
I20260923 22:07:41.523121 3086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "b470ac46f76b4b93ad01b2c1603d9501"
format_stamp: "Formatted at 2026-09-23 22:07:41 on dist-test-slave-b5jz"
server_key: "fe4a545c0d5b6b02fefcd97211a9f029"
server_key_iv: "969d4a2a9fe702567d5752eaaedac3af"
server_key_version: "encryptionkey@0"
I20260923 22:07:41.524426 3086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:07:41.525112 3101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:41.525288 3086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:41.525383 3086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "b470ac46f76b4b93ad01b2c1603d9501"
format_stamp: "Formatted at 2026-09-23 22:07:41 on dist-test-slave-b5jz"
server_key: "fe4a545c0d5b6b02fefcd97211a9f029"
server_key_iv: "969d4a2a9fe702567d5752eaaedac3af"
server_key_version: "encryptionkey@0"
I20260923 22:07:41.525471 3086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:41.547739 3086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:41.550072 3086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:41.550238 3086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:41.554800 3086 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46175
I20260923 22:07:41.554901 3163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46175 every 8 connection(s)
I20260923 22:07:41.555202 3086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:41.555946 3164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:41.558406 3164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501: Bootstrap starting.
I20260923 22:07:41.559000 3164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:41.559296 3164 log.cc:824] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:41.560038 3164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501: No bootstrap required, opened a new log
I20260923 22:07:41.562069 3164 raft_consensus.cc:348] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b470ac46f76b4b93ad01b2c1603d9501" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46175 } }
I20260923 22:07:41.562202 3164 raft_consensus.cc:374] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:41.562225 3164 raft_consensus.cc:729] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b470ac46f76b4b93ad01b2c1603d9501, State: Initialized, Role: FOLLOWER
I20260923 22:07:41.562325 3164 consensus_queue.cc:260] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b470ac46f76b4b93ad01b2c1603d9501" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46175 } }
I20260923 22:07:41.562423 3164 raft_consensus.cc:388] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:41.562453 3164 raft_consensus.cc:482] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:41.562498 3164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:41.563014 3164 raft_consensus.cc:504] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b470ac46f76b4b93ad01b2c1603d9501" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46175 } }
I20260923 22:07:41.563127 3164 leader_election.cc:302] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b470ac46f76b4b93ad01b2c1603d9501; no voters:
I20260923 22:07:41.563313 3164 leader_election.cc:288] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:41.563380 3167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:41.563584 3167 raft_consensus.cc:686] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [term 1 LEADER]: Becoming Leader. State: Replica: b470ac46f76b4b93ad01b2c1603d9501, State: Running, Role: LEADER
I20260923 22:07:41.563619 3164 sys_catalog.cc:564] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:41.563681 3167 consensus_queue.cc:237] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b470ac46f76b4b93ad01b2c1603d9501" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46175 } }
I20260923 22:07:41.564050 3169 sys_catalog.cc:455] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b470ac46f76b4b93ad01b2c1603d9501. Latest consensus state: current_term: 1 leader_uuid: "b470ac46f76b4b93ad01b2c1603d9501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b470ac46f76b4b93ad01b2c1603d9501" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46175 } } }
I20260923 22:07:41.564139 3169 sys_catalog.cc:458] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:41.564038 3168 sys_catalog.cc:455] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b470ac46f76b4b93ad01b2c1603d9501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b470ac46f76b4b93ad01b2c1603d9501" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46175 } } }
I20260923 22:07:41.564287 3168 sys_catalog.cc:458] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:41.564359 3174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:41.564421 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 3086
I20260923 22:07:41.564520 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:07:41.564687 2003 external_mini_cluster.cc:1428] Setting key d4607e7627714128d4d6f3583b83da03
I20260923 22:07:41.564921 3174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:41.566509 3174 catalog_manager.cc:1347] Generated new cluster ID: a1c3738d302c40dfb0d6abf12021ea5c
I20260923 22:07:41.566566 3174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:07:41 dist-test-slave-b5jz krb5kdc[3073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:41.576892 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:41.565247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40242 (local address 127.1.244.254:46175)
0923 22:07:41.565990 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:41.566000 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:41.566032 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:07:41.566152 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:41.566158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:41.566201 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:41.566335 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:41.567160 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.568641 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.569668 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.570405 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.570535 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:41.573305 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:41.573309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:41.573309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:41.573320 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:41.575241 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.575668 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.575671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.575671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.575771 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.575983 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.575983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.575984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.576264 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:41.576411 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:41.576515 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 22:07:41.576654 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":67,"threads_started":1}
I20260923 22:07:41.594965 3174 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:41.595489 3174 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:41.601308 3174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501: Generated new TSK 0
I20260923 22:07:41.601511 3174 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:07:41.626958 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46175
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:41.710503 3190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:41.710747 3190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:41.710789 3190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:41.712028 3190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:41.712081 3190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:41.712112 3190 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:07:41.712145 3190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:07:41.713646 3190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:46175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:41.713912 3190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:41.714118 3190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:41.718833 3196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:41.718878 3198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:41.718852 3195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:41.719265 3190 server_base.cc:1034] running on GCE node
I20260923 22:07:41.719483 3190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:41.719761 3190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:41.720909 3190 hybrid_clock.cc:648] HybridClock initialized: now 1790201261720898 us; error 27 us; skew 500 ppm
Sep 23 22:07:41 dist-test-slave-b5jz krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201261, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:41.723380 3190 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:07:41.723814 3190 webserver.cc:460] Webserver started at http://127.1.244.193:34111/ using document root <none> and password file <none>
I20260923 22:07:41.723986 3190 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:41.724025 3190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:41.724113 3190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:41.725057 3190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "a6ac8e980cd2461fb1c841303418200d"
format_stamp: "Formatted at 2026-09-23 22:07:41 on dist-test-slave-b5jz"
server_key: "dd02520a1f44bf0ec5e2e8082ea0f649"
server_key_iv: "5d41512d9acf64a9e48b18a548e26cc0"
server_key_version: "encryptionkey@0"
I20260923 22:07:41.725335 3190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "a6ac8e980cd2461fb1c841303418200d"
format_stamp: "Formatted at 2026-09-23 22:07:41 on dist-test-slave-b5jz"
server_key: "dd02520a1f44bf0ec5e2e8082ea0f649"
server_key_iv: "5d41512d9acf64a9e48b18a548e26cc0"
server_key_version: "encryptionkey@0"
I20260923 22:07:41.726485 3190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:07:41.727161 3205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:41.727380 3190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:07:41.727478 3190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "a6ac8e980cd2461fb1c841303418200d"
format_stamp: "Formatted at 2026-09-23 22:07:41 on dist-test-slave-b5jz"
server_key: "dd02520a1f44bf0ec5e2e8082ea0f649"
server_key_iv: "5d41512d9acf64a9e48b18a548e26cc0"
server_key_version: "encryptionkey@0"
I20260923 22:07:41.727589 3190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:41.738487 3190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:41.740332 3190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:41.740440 3190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:41.740880 3190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:41.740927 3190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:41.740950 3190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:41.740963 3190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:41.747797 3190 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:45497
I20260923 22:07:41.747928 3320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:45497 every 8 connection(s)
Sep 23 22:07:41 dist-test-slave-b5jz krb5kdc[3073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201261, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:41.748215 3190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:07:41.751286 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:41.741411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:38203 (local address 127.1.244.254:46175)
0923 22:07:41.741487 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:41.741490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:41.741659 (+ 169us) server_negotiation.cc:407] Connection header received
0923 22:07:41.742204 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:41.742206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:41.742226 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:41.742310 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:41.743120 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.744009 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.745572 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.745939 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.746016 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:41.748625 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:41.748627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:41.748627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:41.748637 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:41.750076 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.750486 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.750487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.750487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.750547 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.750761 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.750761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.750761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.750860 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:41.750908 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:41.751159 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 22:07:41.751225 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:07:41.751469 3214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:41.741371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46175 (local address 127.1.244.193:38203)
0923 22:07:41.741529 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:41.741587 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:41.742066 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:41.742376 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:41.742380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:41.742693 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:41.742991 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:41.742998 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.744117 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.744118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:41.745464 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:41.745468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.746040 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.746041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:41.746153 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:41.746657 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:41.746667 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:41.748506 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:41.750167 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:41.750172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:41.750174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:41.750415 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:41.750607 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:41.750607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:41.750608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:41.750706 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:41.750937 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:41.750941 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:41.751048 (+ 107us) client_negotiation.cc:769] Sending connection context
0923 22:07:41.751131 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:07:41.751245 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260923 22:07:41.751857 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 3190
I20260923 22:07:41.751955 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:41.752123 2003 external_mini_cluster.cc:1428] Setting key f7287820356e9524efc8c222048adc63
I20260923 22:07:41.752995 3321 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46175
I20260923 22:07:41.753095 3321 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:41.753343 3321 heartbeater.cc:511] Master 127.1.244.254:46175 requested a full tablet report, sending...
I20260923 22:07:41.754364 3118 ts_manager.cc:194] Registered new tserver with Master: a6ac8e980cd2461fb1c841303418200d (127.1.244.193:45497)
I20260923 22:07:41.755082 3118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:38203
I20260923 22:07:41.759678 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:41.753078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46175 (local address 127.1.244.193:58653)
0923 22:07:41.753489 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:41.753498 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:41.753573 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:41.753703 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:41.753703 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:41.753882 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:41.753946 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:41.753948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.754825 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.754827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:41.755640 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:41.755643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.756135 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.756136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:41.756184 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:41.756614 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:41.756619 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:41.756911 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:41.758722 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:41.758723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:41.758724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:41.758882 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:41.759094 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:41.759095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:41.759095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:41.759144 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:41.759390 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:41.759391 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:41.759423 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:07:41.759473 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:07:41.759598 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":70,"threads_started":1}
I20260923 22:07:41.759732 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:41.753150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:58653 (local address 127.1.244.254:46175)
0923 22:07:41.753193 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:41.753194 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:41.753568 (+ 374us) server_negotiation.cc:407] Connection header received
0923 22:07:41.753633 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:41.753633 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:41.753643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:41.753698 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:41.754034 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.754720 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.755721 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.756053 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.756126 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:41.756996 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:41.756998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:41.756998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:41.757005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:41.758643 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.758957 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.758958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.758959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.759023 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.759203 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.759204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.759204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.759330 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:41.759400 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:41.759499 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 22:07:41.759617 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:07:41.778059 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46175
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:41.857524 3330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:41.857714 3330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:41.857739 3330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:41.858940 3330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:41.858989 3330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:41.859009 3330 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:07:41.859026 3330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:07:41.860464 3330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:46175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:41.860649 3330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:41.860823 3330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:41.865741 3338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:41.865743 3335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:41.865736 3336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:41.865934 3330 server_base.cc:1034] running on GCE node
I20260923 22:07:41.866210 3330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:41.866444 3330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:41.867605 3330 hybrid_clock.cc:648] HybridClock initialized: now 1790201261867594 us; error 36 us; skew 500 ppm
Sep 23 22:07:41 dist-test-slave-b5jz krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201261, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:41.870075 3330 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:07:41.870441 3330 webserver.cc:460] Webserver started at http://127.1.244.194:37175/ using document root <none> and password file <none>
I20260923 22:07:41.870627 3330 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:41.870668 3330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:41.870754 3330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:41.871774 3330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "63d08763497b415bbde0c4f02decb5da"
format_stamp: "Formatted at 2026-09-23 22:07:41 on dist-test-slave-b5jz"
server_key: "a76d4c23bafdab4bd4e3bc2c7ba40080"
server_key_iv: "6553ae07fb74583e3d9cfef7b22a7b4c"
server_key_version: "encryptionkey@0"
I20260923 22:07:41.872069 3330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "63d08763497b415bbde0c4f02decb5da"
format_stamp: "Formatted at 2026-09-23 22:07:41 on dist-test-slave-b5jz"
server_key: "a76d4c23bafdab4bd4e3bc2c7ba40080"
server_key_iv: "6553ae07fb74583e3d9cfef7b22a7b4c"
server_key_version: "encryptionkey@0"
I20260923 22:07:41.873438 3330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:41.874080 3345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:41.874259 3330 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:41.874354 3330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "63d08763497b415bbde0c4f02decb5da"
format_stamp: "Formatted at 2026-09-23 22:07:41 on dist-test-slave-b5jz"
server_key: "a76d4c23bafdab4bd4e3bc2c7ba40080"
server_key_iv: "6553ae07fb74583e3d9cfef7b22a7b4c"
server_key_version: "encryptionkey@0"
I20260923 22:07:41.874408 3330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:41.895740 3330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:41.897521 3330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:41.897631 3330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:41.898057 3330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:41.898100 3330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:41.898123 3330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:41.898136 3330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:41.904811 3330 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:37527
Sep 23 22:07:41 dist-test-slave-b5jz krb5kdc[3073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201261, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:41.904876 3460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:37527 every 8 connection(s)
I20260923 22:07:41.905201 3330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:07:41.907878 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:41.898553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:48137 (local address 127.1.244.254:46175)
0923 22:07:41.898628 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:41.898631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:41.898774 (+ 143us) server_negotiation.cc:407] Connection header received
0923 22:07:41.899265 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:41.899266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:41.899279 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:41.899360 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:41.900238 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.901059 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.902640 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.902954 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.903040 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:41.905472 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:41.905473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:41.905474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:41.905482 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:41.906682 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.907062 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.907063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.907063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.907113 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.907316 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.907317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.907317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.907412 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:41.907459 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:41.907759 (+ 300us) server_negotiation.cc:299] Negotiation successful
0923 22:07:41.907818 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:07:41.908066 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:41.898476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46175 (local address 127.1.244.194:48137)
0923 22:07:41.898646 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:41.898685 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:41.899173 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:41.899449 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:41.899454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:41.899857 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:41.900131 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:41.900138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.901160 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.901161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:41.902507 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:41.902510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.903061 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.903062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:41.903166 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:41.903751 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:41.903758 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:41.905353 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:41.906749 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:41.906755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:41.906756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:41.906991 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:41.907170 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:41.907171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:41.907171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:41.907263 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:41.907465 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:41.907469 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:41.907639 (+ 170us) client_negotiation.cc:769] Sending connection context
0923 22:07:41.907730 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:07:41.907840 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260923 22:07:41.908617 3461 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46175
I20260923 22:07:41.908717 3461 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:41.908875 3461 heartbeater.cc:511] Master 127.1.244.254:46175 requested a full tablet report, sending...
I20260923 22:07:41.909268 3118 ts_manager.cc:194] Registered new tserver with Master: 63d08763497b415bbde0c4f02decb5da (127.1.244.194:37527)
I20260923 22:07:41.909744 3118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:48137
I20260923 22:07:41.912900 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 3330
I20260923 22:07:41.912998 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:41.913187 2003 external_mini_cluster.cc:1428] Setting key 8d47660990d78161fec99606518e2aaa
I20260923 22:07:41.914953 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:41.909597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46175 (local address 127.1.244.194:43917)
0923 22:07:41.909707 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:41.909717 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:41.909775 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:41.909909 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:41.909910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:41.910083 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:41.910129 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:41.910130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.910770 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.910771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:41.911273 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:41.911275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.911706 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.911707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:41.911782 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:41.912105 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:41.912109 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:41.912410 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:41.914059 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:41.914059 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:41.914059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:41.914210 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:41.914396 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:41.914397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:41.914397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:41.914443 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:41.914746 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:41.914747 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:41.914779 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:07:41.914826 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:07:41.914890 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
I20260923 22:07:41.915045 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:41.909659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:43917 (local address 127.1.244.254:46175)
0923 22:07:41.909702 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:41.909704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:41.909768 (+ 64us) server_negotiation.cc:407] Connection header received
0923 22:07:41.909830 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:41.909831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:41.909840 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:41.909890 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:41.910217 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.910679 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.911357 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:41.911638 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:41.911707 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:41.912495 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:41.912497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:41.912497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:41.912504 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:41.913962 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.914277 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.914278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.914278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.914334 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:41.914499 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:41.914499 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:41.914500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:41.914687 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:41.914753 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:41.914849 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:07:41.914921 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:07:41.937644 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46175
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:42.018342 3470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:42.018536 3470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:42.018559 3470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:42.019678 3470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:42.019716 3470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:42.019735 3470 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:07:42.019752 3470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:07:42.021080 3470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:46175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:42.021289 3470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:42.021498 3470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:42.026324 3475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:42.026333 3478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:42.026333 3476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:42.026466 3470 server_base.cc:1034] running on GCE node
I20260923 22:07:42.026759 3470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:42.027034 3470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:42.028201 3470 hybrid_clock.cc:648] HybridClock initialized: now 1790201262028187 us; error 42 us; skew 500 ppm
Sep 23 22:07:42 dist-test-slave-b5jz krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201262, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:42.030834 3470 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:07:42.031205 3470 webserver.cc:460] Webserver started at http://127.1.244.195:37305/ using document root <none> and password file <none>
I20260923 22:07:42.031394 3470 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:42.031433 3470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:42.031616 3470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:42.032625 3470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "e3c6e426c58d4264a8fea2190d0c2f14"
format_stamp: "Formatted at 2026-09-23 22:07:42 on dist-test-slave-b5jz"
server_key: "0bed5c430c38a7e99c270b60789fba2b"
server_key_iv: "bf6a7c7822108582463d98a18592afb9"
server_key_version: "encryptionkey@0"
I20260923 22:07:42.032982 3470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "e3c6e426c58d4264a8fea2190d0c2f14"
format_stamp: "Formatted at 2026-09-23 22:07:42 on dist-test-slave-b5jz"
server_key: "0bed5c430c38a7e99c270b60789fba2b"
server_key_iv: "bf6a7c7822108582463d98a18592afb9"
server_key_version: "encryptionkey@0"
I20260923 22:07:42.034242 3470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:42.034937 3485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:42.035108 3470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:42.035204 3470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "e3c6e426c58d4264a8fea2190d0c2f14"
format_stamp: "Formatted at 2026-09-23 22:07:42 on dist-test-slave-b5jz"
server_key: "0bed5c430c38a7e99c270b60789fba2b"
server_key_iv: "bf6a7c7822108582463d98a18592afb9"
server_key_version: "encryptionkey@0"
I20260923 22:07:42.035285 3470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:42.046427 3470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:42.048691 3470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:42.048851 3470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:42.049335 3470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:42.049400 3470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:42.049435 3470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:42.049480 3470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:07:42 dist-test-slave-b5jz krb5kdc[3073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201262, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:42.056643 3470 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:35605
I20260923 22:07:42.056712 3600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:35605 every 8 connection(s)
I20260923 22:07:42.057096 3470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:07:42.059341 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.049805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:60459 (local address 127.1.244.254:46175)
0923 22:07:42.049878 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.049881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.050060 (+ 179us) server_negotiation.cc:407] Connection header received
0923 22:07:42.050512 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.050513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.050527 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.050612 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:42.051462 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.052358 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.053650 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.053990 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.054075 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.056771 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:42.056773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:42.056773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:42.056781 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:42.058218 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.058698 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.058699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.058699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.058749 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.058907 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.058907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.058908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.058997 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:42.059039 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.059214 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.059272 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:07:42.059525 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.049787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46175 (local address 127.1.244.195:60459)
0923 22:07:42.049926 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.049965 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.050424 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.050662 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.050689 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.051061 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:42.051353 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.051360 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.052459 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.052460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.053547 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.053551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.054099 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.054100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.054243 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.054768 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:42.054781 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:42.056666 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:42.058347 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:42.058352 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:42.058354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:42.058643 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:42.058791 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:42.058791 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:42.058792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:42.058873 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:42.059031 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:42.059034 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:42.059131 (+ 97us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.059202 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.059306 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260923 22:07:42.060103 3601 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46175
I20260923 22:07:42.060205 3601 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:42.060374 3601 heartbeater.cc:511] Master 127.1.244.254:46175 requested a full tablet report, sending...
I20260923 22:07:42.060843 3117 ts_manager.cc:194] Registered new tserver with Master: e3c6e426c58d4264a8fea2190d0c2f14 (127.1.244.195:35605)
I20260923 22:07:42.061371 3117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:60459
I20260923 22:07:42.062001 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 3470
I20260923 22:07:42.062086 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:42.062258 2003 external_mini_cluster.cc:1428] Setting key 21c7766926128dc3b60d214a52b59001
I20260923 22:07:42.063156 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:07:42.067310 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.061160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:34107 (local address 127.1.244.254:46175)
0923 22:07:42.061199 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.061201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.061296 (+ 95us) server_negotiation.cc:407] Connection header received
0923 22:07:42.061345 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.061346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.061354 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.061411 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:42.061764 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.062352 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.063336 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.063628 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.063696 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.064735 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:42.064736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:42.064736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:42.064748 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:42.066244 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.066615 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.066616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.066616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.066662 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.066835 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.066836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.066836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.066919 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:42.066969 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.067144 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.067197 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:07:42.067353 3607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.061107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46175 (local address 127.1.244.195:34107)
0923 22:07:42.061215 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.061237 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.061292 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.061413 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.061414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.061634 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:42.061683 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.061685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.062473 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.062474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.063264 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.063266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.063712 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.063713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.063800 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.064197 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:42.064202 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:42.064662 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:42.066317 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:42.066318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:42.066318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:42.066542 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:42.066716 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:42.066717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:42.066717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:42.066782 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:42.066983 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:42.066984 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:42.067040 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.067118 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.067230 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":63,"threads_started":1}
Sep 23 22:07:42 dist-test-slave-b5jz krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201262, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:07:42 dist-test-slave-b5jz krb5kdc[3073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201262, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:42.095844 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.087996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40250 (local address 127.1.244.254:46175)
0923 22:07:42.088071 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.088073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.088084 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:07:42.088129 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.088130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.088141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.088212 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:42.088994 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.089789 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.091016 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.091232 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.091309 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.093605 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:42.093606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:42.093606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:42.093613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:42.094718 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.095063 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.095064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.095064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.095124 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.095322 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.095323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.095323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.095433 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:42.095533 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.095665 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.095742 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 22:07:42.097344 3117 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:07:42.098863 3117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:42.105168 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.100941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37527 (local address 127.0.0.1:44946)
0923 22:07:42.101190 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.101201 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.101253 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.101501 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.101504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.101507 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:42.101573 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.101575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.103052 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.103053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.103896 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.103899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.104874 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.104875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.104949 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.104953 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.104982 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.105053 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":38,"thread_start_us":50,"threads_started":1}
I20260923 22:07:42.105175 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.101272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.1.244.195:35605)
0923 22:07:42.101436 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.101440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.101599 (+ 159us) server_negotiation.cc:407] Connection header received
0923 22:07:42.101642 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.101644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.101660 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.101739 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:42.101903 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.102968 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.104256 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.104735 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.104832 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.104892 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.105034 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.105074 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260923 22:07:42.105195 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.100941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:45497 (local address 127.0.0.1:46910)
0923 22:07:42.101090 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.101129 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.101188 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.101533 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.101535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.101537 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:42.101584 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.101586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.102570 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.102571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.104093 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.104095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.104939 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.104939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.105000 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.105002 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.105051 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.105089 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":50,"threads_started":1}
I20260923 22:07:42.105391 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.101114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:35605 (local address 127.0.0.1:41496)
0923 22:07:42.101451 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.101460 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.101511 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.101797 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.101798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.101800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:42.101836 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.101838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.103065 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.103066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.104179 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.104181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.104835 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.104836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.104910 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.104925 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.104992 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.105064 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":27,"thread_start_us":38,"threads_started":1}
I20260923 22:07:42.105420 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.101113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.1.244.193:45497)
0923 22:07:42.101367 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.101371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.101379 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:07:42.101424 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.101426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.101442 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.101660 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:42.101734 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.102480 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.104171 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.104848 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.104938 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.104999 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.105263 (+ 264us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.105317 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":73,"threads_started":1}
I20260923 22:07:42.105556 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.101025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44946 (local address 127.1.244.194:37527)
0923 22:07:42.101300 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.101306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.101316 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:07:42.101355 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.101357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.101374 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.101482 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:42.101662 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.102930 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.104028 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.104763 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.105137 (+ 374us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.105203 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.105231 (+ 28us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.105378 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":83,"threads_started":1}
I20260923 22:07:42.106079 3255 tablet_service.cc:1467] Processing CreateTablet for tablet d728696e94204162b0dd2864cee8abde (DEFAULT_TABLE table=test-table [id=41ede0d700f746d08da557c1cab3a26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:42.106226 3395 tablet_service.cc:1467] Processing CreateTablet for tablet d728696e94204162b0dd2864cee8abde (DEFAULT_TABLE table=test-table [id=41ede0d700f746d08da557c1cab3a26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:42.106259 3535 tablet_service.cc:1467] Processing CreateTablet for tablet d728696e94204162b0dd2864cee8abde (DEFAULT_TABLE table=test-table [id=41ede0d700f746d08da557c1cab3a26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:42.106413 3255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d728696e94204162b0dd2864cee8abde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:42.106484 3395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d728696e94204162b0dd2864cee8abde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:42.106498 3535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d728696e94204162b0dd2864cee8abde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:42.109587 3621 tablet_bootstrap.cc:492] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: Bootstrap starting.
I20260923 22:07:42.109643 3622 tablet_bootstrap.cc:492] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: Bootstrap starting.
I20260923 22:07:42.109978 3623 tablet_bootstrap.cc:492] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: Bootstrap starting.
I20260923 22:07:42.110344 3621 tablet_bootstrap.cc:654] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:42.110525 3623 tablet_bootstrap.cc:654] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:42.110689 3622 tablet_bootstrap.cc:654] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:42.110685 3621 log.cc:824] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:42.110879 3623 log.cc:824] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:42.110993 3622 log.cc:824] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:42.111442 3621 tablet_bootstrap.cc:492] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: No bootstrap required, opened a new log
I20260923 22:07:42.111511 3621 ts_tablet_manager.cc:1397] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:42.111708 3622 tablet_bootstrap.cc:492] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: No bootstrap required, opened a new log
I20260923 22:07:42.111771 3622 ts_tablet_manager.cc:1397] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:42.113575 3621 raft_consensus.cc:348] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.113754 3621 raft_consensus.cc:374] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:42.113699 3622 raft_consensus.cc:348] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.113788 3621 raft_consensus.cc:729] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d08763497b415bbde0c4f02decb5da, State: Initialized, Role: FOLLOWER
I20260923 22:07:42.113831 3622 raft_consensus.cc:374] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:42.113869 3622 raft_consensus.cc:729] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c6e426c58d4264a8fea2190d0c2f14, State: Initialized, Role: FOLLOWER
I20260923 22:07:42.113934 3621 consensus_queue.cc:260] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.114024 3622 consensus_queue.cc:260] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.114207 3621 ts_tablet_manager.cc:1428] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:07:42.114223 3461 heartbeater.cc:503] Master 127.1.244.254:46175 was elected leader, sending a full tablet report...
I20260923 22:07:42.114420 3601 heartbeater.cc:503] Master 127.1.244.254:46175 was elected leader, sending a full tablet report...
I20260923 22:07:42.114943 3623 tablet_bootstrap.cc:492] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: No bootstrap required, opened a new log
I20260923 22:07:42.114955 3622 ts_tablet_manager.cc:1428] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:07:42.115010 3623 ts_tablet_manager.cc:1397] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 22:07:42.116489 3623 raft_consensus.cc:348] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.116606 3623 raft_consensus.cc:374] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:42.116626 3623 raft_consensus.cc:729] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ac8e980cd2461fb1c841303418200d, State: Initialized, Role: FOLLOWER
I20260923 22:07:42.116695 3623 consensus_queue.cc:260] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.116884 3623 ts_tablet_manager.cc:1428] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:42.116959 3321 heartbeater.cc:503] Master 127.1.244.254:46175 was elected leader, sending a full tablet report...
I20260923 22:07:42.123988 3629 raft_consensus.cc:482] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:42.124099 3629 raft_consensus.cc:504] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.124382 3629 leader_election.cc:288] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63d08763497b415bbde0c4f02decb5da (127.1.244.194:37527), e3c6e426c58d4264a8fea2190d0c2f14 (127.1.244.195:35605)
I20260923 22:07:42.127748 3214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.124499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37527 (local address 127.1.244.193:54343)
0923 22:07:42.124581 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.124608 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.124656 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.124808 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.124809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.124810 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:42.124872 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.124873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.125684 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.125685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.126805 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.126807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.127470 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.127471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.127581 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.127585 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.127656 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.127698 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260923 22:07:42.127755 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.124570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:54343 (local address 127.1.244.194:37527)
0923 22:07:42.124627 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.124629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.124641 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:07:42.124736 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.124737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.124747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.124799 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:42.124959 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.125611 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.126886 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.127386 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.127458 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.127465 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.127649 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.127693 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:07:42.127993 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.124725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:36901 (local address 127.1.244.195:35605)
0923 22:07:42.124763 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.124765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.124812 (+ 47us) server_negotiation.cc:407] Connection header received
0923 22:07:42.124880 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.124881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.124888 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.124986 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:42.125241 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.125853 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.127048 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.127646 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.127707 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.127714 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.127908 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.127941 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:07:42.128113 3415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d728696e94204162b0dd2864cee8abde" candidate_uuid: "a6ac8e980cd2461fb1c841303418200d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d08763497b415bbde0c4f02decb5da" is_pre_election: true
I20260923 22:07:42.128196 3630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.124634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:35605 (local address 127.1.244.193:36901)
0923 22:07:42.124759 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.124768 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.124823 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.125008 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.125009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.125010 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:42.125153 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.125156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.125930 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.125931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.126981 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.126983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.127745 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.127746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.127822 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.127826 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.128069 (+ 243us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.128114 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
I20260923 22:07:42.128253 3415 raft_consensus.cc:2454] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ac8e980cd2461fb1c841303418200d in term 0.
I20260923 22:07:42.128476 3207 leader_election.cc:302] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63d08763497b415bbde0c4f02decb5da, a6ac8e980cd2461fb1c841303418200d; no voters:
I20260923 22:07:42.128630 3629 raft_consensus.cc:2781] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:42.128607 3555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d728696e94204162b0dd2864cee8abde" candidate_uuid: "a6ac8e980cd2461fb1c841303418200d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" is_pre_election: true
I20260923 22:07:42.128674 3629 raft_consensus.cc:482] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:42.128702 3629 raft_consensus.cc:3037] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:42.128716 3555 raft_consensus.cc:2454] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ac8e980cd2461fb1c841303418200d in term 0.
I20260923 22:07:42.129436 3629 raft_consensus.cc:504] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.129619 3629 leader_election.cc:288] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [CANDIDATE]: Term 1 election: Requested vote from peers 63d08763497b415bbde0c4f02decb5da (127.1.244.194:37527), e3c6e426c58d4264a8fea2190d0c2f14 (127.1.244.195:35605)
I20260923 22:07:42.129746 3415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d728696e94204162b0dd2864cee8abde" candidate_uuid: "a6ac8e980cd2461fb1c841303418200d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d08763497b415bbde0c4f02decb5da"
I20260923 22:07:42.129783 3555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d728696e94204162b0dd2864cee8abde" candidate_uuid: "a6ac8e980cd2461fb1c841303418200d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c6e426c58d4264a8fea2190d0c2f14"
I20260923 22:07:42.129820 3415 raft_consensus.cc:3037] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:42.129873 3555 raft_consensus.cc:3037] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:42.130677 3555 raft_consensus.cc:2454] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ac8e980cd2461fb1c841303418200d in term 1.
I20260923 22:07:42.130714 3415 raft_consensus.cc:2454] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ac8e980cd2461fb1c841303418200d in term 1.
I20260923 22:07:42.130924 3207 leader_election.cc:302] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ac8e980cd2461fb1c841303418200d, e3c6e426c58d4264a8fea2190d0c2f14; no voters:
I20260923 22:07:42.131012 3629 raft_consensus.cc:2781] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:42.131145 3629 raft_consensus.cc:686] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 1 LEADER]: Becoming Leader. State: Replica: a6ac8e980cd2461fb1c841303418200d, State: Running, Role: LEADER
I20260923 22:07:42.131242 3629 consensus_queue.cc:237] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.131932 3117 catalog_manager.cc:5495] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ac8e980cd2461fb1c841303418200d (127.1.244.193). New cstate: current_term: 1 leader_uuid: "a6ac8e980cd2461fb1c841303418200d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } health_report { overall_health: UNKNOWN } } }
I20260923 22:07:42.143209 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.140546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46912 (local address 127.1.244.193:45497)
0923 22:07:42.140586 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.140588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.140596 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:07:42.140685 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.140686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.140704 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.140766 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:42.140942 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.141384 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.142180 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.142381 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.142470 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.142692 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:42.142801 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:42.142890 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.142937 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.142954 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:07:42.144829 3415 raft_consensus.cc:1261] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [term 1 FOLLOWER]: Refusing update from remote peer a6ac8e980cd2461fb1c841303418200d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:42.144848 3555 raft_consensus.cc:1261] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Refusing update from remote peer a6ac8e980cd2461fb1c841303418200d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:42.145124 3631 consensus_queue.cc:1035] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:42.145241 3629 consensus_queue.cc:1035] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [LEADER]: Connected to new peer: Peer: permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:42.150980 3118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:42.151073 3118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:42.151880 3118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501: Sending DeleteTablet for 3 replicas of tablet d728696e94204162b0dd2864cee8abde
I20260923 22:07:42.152194 3255 tablet_service.cc:1514] Processing DeleteTablet for tablet d728696e94204162b0dd2864cee8abde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:42 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46910
I20260923 22:07:42.152237 3535 tablet_service.cc:1514] Processing DeleteTablet for tablet d728696e94204162b0dd2864cee8abde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:42 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:41496
I20260923 22:07:42.152401 3395 tablet_service.cc:1514] Processing DeleteTablet for tablet d728696e94204162b0dd2864cee8abde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:42 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44946
I20260923 22:07:42.152444 3645 tablet_replica.cc:330] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: stopping tablet replica
I20260923 22:07:42.152419 3118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:07:42.152567 3645 raft_consensus.cc:2229] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260923 22:07:42.152571 3118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:42.152694 3645 raft_consensus.cc:2258] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:42.152781 3644 tablet_replica.cc:330] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: stopping tablet replica
I20260923 22:07:42.152984 3644 raft_consensus.cc:2229] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:07:42.153107 3644 raft_consensus.cc:2258] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:42.153100 3645 ts_tablet_manager.cc:1905] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:42.153519 3644 ts_tablet_manager.cc:1905] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:42.154230 3255 tablet_service.cc:1467] Processing CreateTablet for tablet 3c7fa2456c44482eb30f61101b117e49 (DEFAULT_TABLE table=test-table [id=28e1d663cb3a44e6bd637386db2cc71b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:42.154304 3535 tablet_service.cc:1467] Processing CreateTablet for tablet 3c7fa2456c44482eb30f61101b117e49 (DEFAULT_TABLE table=test-table [id=28e1d663cb3a44e6bd637386db2cc71b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:42.154337 3255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c7fa2456c44482eb30f61101b117e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:42.154472 3535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c7fa2456c44482eb30f61101b117e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:42.155282 3645 ts_tablet_manager.cc:1918] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:07:42.155340 3645 log.cc:1196] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/d728696e94204162b0dd2864cee8abde
I20260923 22:07:42.155646 3395 tablet_service.cc:1467] Processing CreateTablet for tablet 3c7fa2456c44482eb30f61101b117e49 (DEFAULT_TABLE table=test-table [id=28e1d663cb3a44e6bd637386db2cc71b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:42.155681 3623 tablet_bootstrap.cc:492] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: Bootstrap starting.
I20260923 22:07:42.155773 3395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c7fa2456c44482eb30f61101b117e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:42.156097 3623 tablet_bootstrap.cc:654] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:42.156698 3622 tablet_bootstrap.cc:492] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: Bootstrap starting.
I20260923 22:07:42.156720 3646 tablet_replica.cc:330] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: stopping tablet replica
I20260923 22:07:42.155622 3645 ts_tablet_manager.cc:1939] T d728696e94204162b0dd2864cee8abde P e3c6e426c58d4264a8fea2190d0c2f14: Deleting consensus metadata
I20260923 22:07:42.156961 3646 raft_consensus.cc:2229] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:42.157119 3646 raft_consensus.cc:2258] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:42.157173 3622 tablet_bootstrap.cc:654] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:42.157591 3103 catalog_manager.cc:4841] TS e3c6e426c58d4264a8fea2190d0c2f14 (127.1.244.195:35605): tablet d728696e94204162b0dd2864cee8abde (table test-table [id=41ede0d700f746d08da557c1cab3a26f]) successfully deleted
I20260923 22:07:42.157768 3622 tablet_bootstrap.cc:492] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: No bootstrap required, opened a new log
I20260923 22:07:42.157809 3622 ts_tablet_manager.cc:1397] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:42.157948 3646 ts_tablet_manager.cc:1905] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:42.157967 3622 raft_consensus.cc:348] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.158026 3622 raft_consensus.cc:374] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:42.158052 3622 raft_consensus.cc:729] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c6e426c58d4264a8fea2190d0c2f14, State: Initialized, Role: FOLLOWER
I20260923 22:07:42.158100 3622 consensus_queue.cc:260] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.158197 3622 ts_tablet_manager.cc:1428] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:42.158983 3621 tablet_bootstrap.cc:492] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: Bootstrap starting.
I20260923 22:07:42.159371 3621 tablet_bootstrap.cc:654] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:42.159998 3621 tablet_bootstrap.cc:492] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: No bootstrap required, opened a new log
I20260923 22:07:42.160061 3621 ts_tablet_manager.cc:1397] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:42.160121 3646 ts_tablet_manager.cc:1918] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:07:42.160192 3646 log.cc:1196] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/d728696e94204162b0dd2864cee8abde
I20260923 22:07:42.160193 3621 raft_consensus.cc:348] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.160271 3621 raft_consensus.cc:374] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:42.160301 3621 raft_consensus.cc:729] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d08763497b415bbde0c4f02decb5da, State: Initialized, Role: FOLLOWER
I20260923 22:07:42.160346 3621 consensus_queue.cc:260] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.160427 3621 ts_tablet_manager.cc:1428] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:42.160521 3646 ts_tablet_manager.cc:1939] T d728696e94204162b0dd2864cee8abde P 63d08763497b415bbde0c4f02decb5da: Deleting consensus metadata
I20260923 22:07:42.160897 3103 catalog_manager.cc:4841] TS 63d08763497b415bbde0c4f02decb5da (127.1.244.194:37527): tablet d728696e94204162b0dd2864cee8abde (table test-table [id=41ede0d700f746d08da557c1cab3a26f]) successfully deleted
I20260923 22:07:42.161875 3644 ts_tablet_manager.cc:1918] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:07:42.161953 3644 log.cc:1196] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/d728696e94204162b0dd2864cee8abde
I20260923 22:07:42.162225 3644 ts_tablet_manager.cc:1939] T d728696e94204162b0dd2864cee8abde P a6ac8e980cd2461fb1c841303418200d: Deleting consensus metadata
I20260923 22:07:42.162616 3104 catalog_manager.cc:4841] TS a6ac8e980cd2461fb1c841303418200d (127.1.244.193:45497): tablet d728696e94204162b0dd2864cee8abde (table test-table [id=41ede0d700f746d08da557c1cab3a26f]) successfully deleted
I20260923 22:07:42.163933 3623 tablet_bootstrap.cc:492] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: No bootstrap required, opened a new log
I20260923 22:07:42.163988 3623 ts_tablet_manager.cc:1397] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260923 22:07:42.164121 3623 raft_consensus.cc:348] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.164199 3623 raft_consensus.cc:374] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:42.164220 3623 raft_consensus.cc:729] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ac8e980cd2461fb1c841303418200d, State: Initialized, Role: FOLLOWER
I20260923 22:07:42.164268 3623 consensus_queue.cc:260] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.164353 3623 ts_tablet_manager.cc:1428] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:42.224624 3627 raft_consensus.cc:482] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:42.224752 3627 raft_consensus.cc:504] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.225107 3627 leader_election.cc:288] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ac8e980cd2461fb1c841303418200d (127.1.244.193:45497), e3c6e426c58d4264a8fea2190d0c2f14 (127.1.244.195:35605)
I20260923 22:07:42.228926 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.225257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:45497 (local address 127.1.244.194:44771)
0923 22:07:42.225322 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.225334 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.225384 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.225712 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.225713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.225715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:42.225830 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.225831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.226783 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.226784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.227927 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.227929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.228674 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.228674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.228740 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.228744 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.228794 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.228835 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":44}
I20260923 22:07:42.228940 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.225467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:44771 (local address 127.1.244.193:45497)
0923 22:07:42.225540 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.225542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.225550 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:07:42.225577 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.225579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.225591 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.225639 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:42.225934 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.226663 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.228042 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.228565 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.228643 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.228652 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.228816 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.228854 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:07:42.229476 3275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c7fa2456c44482eb30f61101b117e49" candidate_uuid: "63d08763497b415bbde0c4f02decb5da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ac8e980cd2461fb1c841303418200d" is_pre_election: true
I20260923 22:07:42.229610 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.225359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:55501 (local address 127.1.244.195:35605)
0923 22:07:42.225402 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.225404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.225587 (+ 183us) server_negotiation.cc:407] Connection header received
0923 22:07:42.225621 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.225622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.225631 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.225699 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:42.226798 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.227521 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.228637 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.229225 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.229301 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.229310 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.229500 (+ 190us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.229547 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:07:42.229625 3275 raft_consensus.cc:2454] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63d08763497b415bbde0c4f02decb5da in term 0.
I20260923 22:07:42.229856 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.225275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:35605 (local address 127.1.244.194:55501)
0923 22:07:42.225504 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.225514 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.225572 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.225712 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.225713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.225715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:42.225830 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.225831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.227623 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.227624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.228553 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.228556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.229313 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.229314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.229390 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.229394 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.229715 (+ 321us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.229758 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":4,"thread_start_us":58,"threads_started":1}
I20260923 22:07:42.229868 3348 leader_election.cc:302] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63d08763497b415bbde0c4f02decb5da, a6ac8e980cd2461fb1c841303418200d; no voters:
I20260923 22:07:42.230021 3627 raft_consensus.cc:2781] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:42.230073 3627 raft_consensus.cc:482] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:42.230093 3627 raft_consensus.cc:3037] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:42.230118 3555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c7fa2456c44482eb30f61101b117e49" candidate_uuid: "63d08763497b415bbde0c4f02decb5da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" is_pre_election: true
I20260923 22:07:42.230202 3555 raft_consensus.cc:2454] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63d08763497b415bbde0c4f02decb5da in term 0.
I20260923 22:07:42.231009 3627 raft_consensus.cc:504] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.231186 3627 leader_election.cc:288] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [CANDIDATE]: Term 1 election: Requested vote from peers a6ac8e980cd2461fb1c841303418200d (127.1.244.193:45497), e3c6e426c58d4264a8fea2190d0c2f14 (127.1.244.195:35605)
I20260923 22:07:42.231334 3555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c7fa2456c44482eb30f61101b117e49" candidate_uuid: "63d08763497b415bbde0c4f02decb5da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c6e426c58d4264a8fea2190d0c2f14"
I20260923 22:07:42.231341 3275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c7fa2456c44482eb30f61101b117e49" candidate_uuid: "63d08763497b415bbde0c4f02decb5da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ac8e980cd2461fb1c841303418200d"
I20260923 22:07:42.231407 3555 raft_consensus.cc:3037] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:42.231415 3275 raft_consensus.cc:3037] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:42.232613 3555 raft_consensus.cc:2454] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63d08763497b415bbde0c4f02decb5da in term 1.
I20260923 22:07:42.232613 3275 raft_consensus.cc:2454] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63d08763497b415bbde0c4f02decb5da in term 1.
I20260923 22:07:42.232780 3347 leader_election.cc:302] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63d08763497b415bbde0c4f02decb5da, e3c6e426c58d4264a8fea2190d0c2f14; no voters:
I20260923 22:07:42.232879 3627 raft_consensus.cc:2781] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:42.233021 3627 raft_consensus.cc:686] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 1 LEADER]: Becoming Leader. State: Replica: 63d08763497b415bbde0c4f02decb5da, State: Running, Role: LEADER
I20260923 22:07:42.233119 3627 consensus_queue.cc:237] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.233829 3118 catalog_manager.cc:5495] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da reported cstate change: term changed from 0 to 1, leader changed from <none> to 63d08763497b415bbde0c4f02decb5da (127.1.244.194). New cstate: current_term: 1 leader_uuid: "63d08763497b415bbde0c4f02decb5da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } health_report { overall_health: UNKNOWN } } }
W20260923 22:07:42.248929 3322 tablet.cc:2369] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 22:07:42 dist-test-slave-b5jz krb5kdc[3073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201261, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:42.271142 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.264291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.1.244.254:46175)
0923 22:07:42.264374 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.264376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.264508 (+ 132us) server_negotiation.cc:407] Connection header received
0923 22:07:42.264586 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.264587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.264597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.264665 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:42.265153 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.265646 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.266413 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.266612 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.266708 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.268834 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:42.268836 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:42.268836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:42.268842 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:42.270048 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.270406 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.270407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.270407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.270454 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.270663 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.270664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.270664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.270763 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:42.270812 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.270999 (+ 187us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.271077 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:07:42.271291 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.264243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46175 (local address 127.0.0.1:40254)
0923 22:07:42.264433 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.264443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.264526 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.264691 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.264692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.264997 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:42.265041 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.265044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.265737 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.265737 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.266324 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.266326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.266708 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.266709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.266760 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.267326 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:42.267331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:42.268747 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:42.270128 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:42.270134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:42.270135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:42.270327 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:42.270536 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:42.270537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:42.270538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:42.270601 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:42.270823 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:42.270828 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:42.270878 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.270978 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.271153 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":70,"thread_start_us":104,"threads_started":1}
I20260923 22:07:42.273236 3118 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:40254:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:07:42.273411 3118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:42.274860 3255 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6de6e412dd4f5893f1b958ec686bad (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cfc8db46e8a84d8d8357e2b2ddd93032]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:07:42.274899 3395 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6de6e412dd4f5893f1b958ec686bad (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cfc8db46e8a84d8d8357e2b2ddd93032]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:07:42.274911 3535 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6de6e412dd4f5893f1b958ec686bad (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cfc8db46e8a84d8d8357e2b2ddd93032]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:07:42.275022 3255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6de6e412dd4f5893f1b958ec686bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:42.275022 3395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6de6e412dd4f5893f1b958ec686bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:42.275069 3535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6de6e412dd4f5893f1b958ec686bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:42.276510 3622 tablet_bootstrap.cc:492] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: Bootstrap starting.
I20260923 22:07:42.276580 3623 tablet_bootstrap.cc:492] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: Bootstrap starting.
I20260923 22:07:42.276897 3623 tablet_bootstrap.cc:654] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:42.276960 3622 tablet_bootstrap.cc:654] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:42.277307 3621 tablet_bootstrap.cc:492] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: Bootstrap starting.
I20260923 22:07:42.277487 3623 tablet_bootstrap.cc:492] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: No bootstrap required, opened a new log
I20260923 22:07:42.277546 3623 ts_tablet_manager.cc:1397] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:42.277649 3623 raft_consensus.cc:348] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.277716 3623 raft_consensus.cc:374] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:42.277731 3623 raft_consensus.cc:729] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ac8e980cd2461fb1c841303418200d, State: Initialized, Role: FOLLOWER
I20260923 22:07:42.277702 3621 tablet_bootstrap.cc:654] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:42.277787 3623 consensus_queue.cc:260] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.277884 3623 ts_tablet_manager.cc:1428] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:42.278106 3640 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.278219 3640 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:42.278527 3621 tablet_bootstrap.cc:492] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: No bootstrap required, opened a new log
I20260923 22:07:42.278589 3621 ts_tablet_manager.cc:1397] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:42.278736 3621 raft_consensus.cc:348] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.278826 3621 raft_consensus.cc:374] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:42.278847 3621 raft_consensus.cc:729] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d08763497b415bbde0c4f02decb5da, State: Initialized, Role: FOLLOWER
I20260923 22:07:42.278906 3621 consensus_queue.cc:260] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.279009 3621 ts_tablet_manager.cc:1428] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:42.279063 3627 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.279142 3627 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:42.279211 3622 tablet_bootstrap.cc:492] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: No bootstrap required, opened a new log
I20260923 22:07:42.279253 3622 ts_tablet_manager.cc:1397] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:07:42.279388 3622 raft_consensus.cc:348] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.279474 3622 raft_consensus.cc:374] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:42.279500 3622 raft_consensus.cc:729] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c6e426c58d4264a8fea2190d0c2f14, State: Initialized, Role: FOLLOWER
I20260923 22:07:42.279578 3622 consensus_queue.cc:260] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.279693 3622 ts_tablet_manager.cc:1428] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:42.279769 3628 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.279873 3628 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:42.303292 3640 raft_consensus.cc:482] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:42.303387 3640 raft_consensus.cc:504] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.303598 3640 leader_election.cc:288] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63d08763497b415bbde0c4f02decb5da (127.1.244.194:37527), e3c6e426c58d4264a8fea2190d0c2f14 (127.1.244.195:35605)
I20260923 22:07:42.303768 3415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6de6e412dd4f5893f1b958ec686bad" candidate_uuid: "a6ac8e980cd2461fb1c841303418200d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d08763497b415bbde0c4f02decb5da" is_pre_election: true
I20260923 22:07:42.303791 3555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6de6e412dd4f5893f1b958ec686bad" candidate_uuid: "a6ac8e980cd2461fb1c841303418200d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" is_pre_election: true
I20260923 22:07:42.303865 3415 raft_consensus.cc:2454] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ac8e980cd2461fb1c841303418200d in term 0.
I20260923 22:07:42.303872 3555 raft_consensus.cc:2454] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ac8e980cd2461fb1c841303418200d in term 0.
I20260923 22:07:42.304013 3207 leader_election.cc:302] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63d08763497b415bbde0c4f02decb5da, a6ac8e980cd2461fb1c841303418200d; no voters:
I20260923 22:07:42.304113 3640 raft_consensus.cc:2781] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:42.304158 3640 raft_consensus.cc:482] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:42.304177 3640 raft_consensus.cc:3037] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:42.304828 3640 raft_consensus.cc:504] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.304953 3640 leader_election.cc:288] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [CANDIDATE]: Term 1 election: Requested vote from peers 63d08763497b415bbde0c4f02decb5da (127.1.244.194:37527), e3c6e426c58d4264a8fea2190d0c2f14 (127.1.244.195:35605)
I20260923 22:07:42.305088 3415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6de6e412dd4f5893f1b958ec686bad" candidate_uuid: "a6ac8e980cd2461fb1c841303418200d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d08763497b415bbde0c4f02decb5da"
I20260923 22:07:42.305162 3415 raft_consensus.cc:3037] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:42.305128 3555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6de6e412dd4f5893f1b958ec686bad" candidate_uuid: "a6ac8e980cd2461fb1c841303418200d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c6e426c58d4264a8fea2190d0c2f14"
I20260923 22:07:42.305210 3555 raft_consensus.cc:3037] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:42.305778 3555 raft_consensus.cc:2454] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ac8e980cd2461fb1c841303418200d in term 1.
I20260923 22:07:42.305991 3207 leader_election.cc:302] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ac8e980cd2461fb1c841303418200d, e3c6e426c58d4264a8fea2190d0c2f14; no voters:
I20260923 22:07:42.306123 3415 raft_consensus.cc:2454] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ac8e980cd2461fb1c841303418200d in term 1.
I20260923 22:07:42.306113 3640 raft_consensus.cc:2781] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:42.306185 3640 raft_consensus.cc:686] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [term 1 LEADER]: Becoming Leader. State: Replica: a6ac8e980cd2461fb1c841303418200d, State: Running, Role: LEADER
I20260923 22:07:42.306233 3640 consensus_queue.cc:237] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } }
I20260923 22:07:42.306509 3629 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: TxnStatusTablet state changed. Reason: New leader a6ac8e980cd2461fb1c841303418200d. Latest consensus state: current_term: 1 leader_uuid: "a6ac8e980cd2461fb1c841303418200d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.306604 3629 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:07:42.306727 3663 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:07:42.306825 3663 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:07:42.306775 3118 catalog_manager.cc:5495] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ac8e980cd2461fb1c841303418200d (127.1.244.193). New cstate: current_term: 1 leader_uuid: "a6ac8e980cd2461fb1c841303418200d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } health_report { overall_health: UNKNOWN } } }
W20260923 22:07:42.307647 3602 tablet.cc:2369] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:07:42.315569 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.313038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46914 (local address 127.1.244.193:45497)
0923 22:07:42.313081 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.313083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.313097 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:07:42.313161 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.313162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.313173 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.313236 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:42.313324 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.314052 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.314768 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.315078 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.315150 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.315313 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:42.315347 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:42.315400 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.315458 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.315479 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:07:42.315587 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.312966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:45497 (local address 127.0.0.1:46914)
0923 22:07:42.313045 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.313059 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.313109 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.313242 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.313243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.313244 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:07:42.313286 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.313288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.314135 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.314135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.314692 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.314693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.315163 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.315164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.315218 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.315232 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:07:42.315410 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:07:42.315417 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.315453 (+ 36us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.315493 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 22:07:42.316666 3555 raft_consensus.cc:1261] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Refusing update from remote peer a6ac8e980cd2461fb1c841303418200d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:42.316684 3415 raft_consensus.cc:1261] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da [term 1 FOLLOWER]: Refusing update from remote peer a6ac8e980cd2461fb1c841303418200d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:42.316874 3640 consensus_queue.cc:1035] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:42.316987 3640 consensus_queue.cc:1035] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d [LEADER]: Connected to new peer: Peer: permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:42.317487 3627 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: TxnStatusTablet state changed. Reason: New leader a6ac8e980cd2461fb1c841303418200d. Latest consensus state: current_term: 1 leader_uuid: "a6ac8e980cd2461fb1c841303418200d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.317480 3628 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: TxnStatusTablet state changed. Reason: New leader a6ac8e980cd2461fb1c841303418200d. Latest consensus state: current_term: 1 leader_uuid: "a6ac8e980cd2461fb1c841303418200d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.317569 3627 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:42.317576 3628 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:42.318323 3640 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6ac8e980cd2461fb1c841303418200d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.318403 3640 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:07:42.318565 3634 mvcc.cc:205] Tried to move back new op lower bound from 7332664370447454208 to 7332664370406735872. Current Snapshot: MvccSnapshot[applied={T|T < 7332664370447454208 or (T in {7332664370447454208})}]
I20260923 22:07:42.319484 3628 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6ac8e980cd2461fb1c841303418200d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.319586 3628 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P e3c6e426c58d4264a8fea2190d0c2f14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:42.322676 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.320197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44948 (local address 127.1.244.194:37527)
0923 22:07:42.320230 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.320232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.320269 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:07:42.320350 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.320350 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.320365 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.320418 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:42.320542 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.320991 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.321985 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.322159 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.322210 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.322356 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:42.322451 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:42.322500 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.322562 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.322587 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:07:42.324103 3629 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6ac8e980cd2461fb1c841303418200d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.324188 3629 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P a6ac8e980cd2461fb1c841303418200d: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:07:42.324607 3636 mvcc.cc:205] Tried to move back new op lower bound from 7332664370447454208 to 7332664370406735872. Current Snapshot: MvccSnapshot[applied={T|T < 7332664370447454208}]
I20260923 22:07:42.324705 3627 tablet_replica.cc:439] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6ac8e980cd2461fb1c841303418200d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 } } peers { permanent_uuid: "63d08763497b415bbde0c4f02decb5da" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37527 } } peers { permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 } } }
I20260923 22:07:42.324782 3627 tablet_replica.cc:442] T 1f6de6e412dd4f5893f1b958ec686bad P 63d08763497b415bbde0c4f02decb5da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:42.327963 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.325525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:40167 (local address 127.1.244.193:45497)
0923 22:07:42.325561 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.325563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.325571 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:07:42.325623 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.325623 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.325632 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.325672 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:42.325796 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.326225 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.327289 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.327477 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.327538 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.327730 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:42.327762 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:42.327804 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.327896 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.327915 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:07:42.328009 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.325455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:45497 (local address 127.1.244.194:40167)
0923 22:07:42.325522 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.325532 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.325569 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.325684 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.325684 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.325685 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:07:42.325742 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.325744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.326297 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.326298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.327206 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.327208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.327560 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.327561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.327630 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.327646 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:07:42.327824 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:07:42.327826 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.327893 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.327937 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:07:42.330099 3555 raft_consensus.cc:1261] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Refusing update from remote peer 63d08763497b415bbde0c4f02decb5da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:42.330170 3275 raft_consensus.cc:1261] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [term 1 FOLLOWER]: Refusing update from remote peer 63d08763497b415bbde0c4f02decb5da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:42.330344 3651 consensus_queue.cc:1035] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c6e426c58d4264a8fea2190d0c2f14" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:42.330444 3627 consensus_queue.cc:1035] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ac8e980cd2461fb1c841303418200d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:42.339306 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.336353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53745 (local address 127.1.244.194:37527)
0923 22:07:42.336393 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.336395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.336829 (+ 434us) server_negotiation.cc:407] Connection header received
0923 22:07:42.336908 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.336909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.336918 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.336973 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:42.337103 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.337736 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.338647 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.338848 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.338926 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.339090 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:42.339122 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:42.339170 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.339222 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.339240 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:07:42.339367 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.336167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37527 (local address 127.1.244.193:53745)
0923 22:07:42.336336 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:42.336348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:42.336858 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:42.336977 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:42.336978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:42.336980 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:07:42.337035 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.337036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.337797 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.337798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.338572 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:42.338575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.338929 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.338930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:42.338985 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.339000 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:07:42.339170 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:07:42.339173 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:07:42.339219 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:07:42.339263 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260923 22:07:42.347747 3117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:42.347851 3117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:42.348668 3117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b470ac46f76b4b93ad01b2c1603d9501: Sending DeleteTablet for 3 replicas of tablet 3c7fa2456c44482eb30f61101b117e49
I20260923 22:07:42.348894 3255 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c7fa2456c44482eb30f61101b117e49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:42 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46910
I20260923 22:07:42.348985 3644 tablet_replica.cc:330] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: stopping tablet replica
I20260923 22:07:42.349002 3395 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c7fa2456c44482eb30f61101b117e49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:42 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44946
I20260923 22:07:42.349030 3644 raft_consensus.cc:2229] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:42.349072 3644 raft_consensus.cc:2258] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:42.349107 3535 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c7fa2456c44482eb30f61101b117e49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:42 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:41496
I20260923 22:07:42.349195 3646 tablet_replica.cc:330] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: stopping tablet replica
I20260923 22:07:42.349239 3646 raft_consensus.cc:2229] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:07:42.349325 3646 raft_consensus.cc:2258] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:42.349395 3645 tablet_replica.cc:330] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: stopping tablet replica
I20260923 22:07:42.349450 3645 raft_consensus.cc:2229] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:42.349485 3645 raft_consensus.cc:2258] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:42.350085 3646 ts_tablet_manager.cc:1905] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:42.350093 3645 ts_tablet_manager.cc:1905] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:42.351759 3645 ts_tablet_manager.cc:1918] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:07:42.351817 3645 log.cc:1196] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/3c7fa2456c44482eb30f61101b117e49
I20260923 22:07:42.352061 3645 ts_tablet_manager.cc:1939] T 3c7fa2456c44482eb30f61101b117e49 P e3c6e426c58d4264a8fea2190d0c2f14: Deleting consensus metadata
I20260923 22:07:42.353124 3103 catalog_manager.cc:4841] TS e3c6e426c58d4264a8fea2190d0c2f14 (127.1.244.195:35605): tablet 3c7fa2456c44482eb30f61101b117e49 (table test-table [id=28e1d663cb3a44e6bd637386db2cc71b]) successfully deleted
I20260923 22:07:42.353386 3646 ts_tablet_manager.cc:1918] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:07:42.353454 3646 log.cc:1196] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/3c7fa2456c44482eb30f61101b117e49
I20260923 22:07:42.353680 3646 ts_tablet_manager.cc:1939] T 3c7fa2456c44482eb30f61101b117e49 P 63d08763497b415bbde0c4f02decb5da: Deleting consensus metadata
I20260923 22:07:42.353971 3103 catalog_manager.cc:4841] TS 63d08763497b415bbde0c4f02decb5da (127.1.244.194:37527): tablet 3c7fa2456c44482eb30f61101b117e49 (table test-table [id=28e1d663cb3a44e6bd637386db2cc71b]) successfully deleted
I20260923 22:07:42.354310 3644 ts_tablet_manager.cc:1905] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:42.355816 3644 ts_tablet_manager.cc:1918] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:07:42.355877 3644 log.cc:1196] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/3c7fa2456c44482eb30f61101b117e49
I20260923 22:07:42.356089 3644 ts_tablet_manager.cc:1939] T 3c7fa2456c44482eb30f61101b117e49 P a6ac8e980cd2461fb1c841303418200d: Deleting consensus metadata
I20260923 22:07:42.356454 3104 catalog_manager.cc:4841] TS a6ac8e980cd2461fb1c841303418200d (127.1.244.193:45497): tablet 3c7fa2456c44482eb30f61101b117e49 (table test-table [id=28e1d663cb3a44e6bd637386db2cc71b]) successfully deleted
Sep 23 22:07:42 dist-test-slave-b5jz krb5kdc[3073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201262, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:07:42.369221 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.353201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46928 (local address 127.1.244.193:45497)
0923 22:07:42.353256 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.353258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.359079 (+ 5821us) server_negotiation.cc:407] Connection header received
0923 22:07:42.359141 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.359142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.359154 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.359220 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:42.359929 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.360689 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.361862 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.362171 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.362240 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.365947 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:42.365957 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:42.365957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:42.365964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:42.367965 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.368371 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.368374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.368375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.368439 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.368651 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.368652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.368652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.368818 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:42.368951 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.369046 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.369126 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260923 22:07:42.369423 3315 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46928
I20260923 22:07:42.377998 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:42.370273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.1.244.254:46175)
0923 22:07:42.370313 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:42.370315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:42.370546 (+ 231us) server_negotiation.cc:407] Connection header received
0923 22:07:42.370648 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:42.370649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:42.370662 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:42.370724 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:42.371346 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.372169 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.373165 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:42.373486 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:42.373558 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:42.374964 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:42.374966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:42.374966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:42.374973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:42.376710 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.377133 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.377134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.377135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.377194 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:42.377404 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:42.377405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:42.377406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:42.377532 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:42.377626 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:42.377820 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 22:07:42.377900 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:07:42.378439 3117 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40258
I20260923 22:07:42.379284 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 3190
I20260923 22:07:42.386058 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 3330
I20260923 22:07:42.391464 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 3470
I20260923 22:07:42.395853 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 3086
2026-09-23T22:07:42Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2103 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:42 dist-test-slave-b5jz krb5kdc[3681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:42 dist-test-slave-b5jz krb5kdc[3681](info): set up 2 sockets
Sep 23 22:07:42 dist-test-slave-b5jz krb5kdc[3681](info): commencing operation
krb5kdc: starting...
W20260923 22:07:43.435266 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:07:43 dist-test-slave-b5jz krb5kdc[3681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:07:43.547266 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39595
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:39595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:43.625272 3694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:43.625519 3694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:43.625557 3694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:43.626688 3694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:43.626754 3694 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:07:43.626778 3694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:43.626811 3694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:43.626837 3694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:43.626863 3694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:43.628400 3694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:39595
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:43.628671 3694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:43.628888 3694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:43.633796 3699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:43.633788 3702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:43.633844 3700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:43.634342 3694 server_base.cc:1034] running on GCE node
I20260923 22:07:43.634614 3694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:43.634909 3694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:43.636098 3694 hybrid_clock.cc:648] HybridClock initialized: now 1790201263636083 us; error 27 us; skew 500 ppm
Sep 23 22:07:43 dist-test-slave-b5jz krb5kdc[3681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201263, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:43.643314 3694 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:07:43.643772 3694 webserver.cc:460] Webserver started at http://127.1.244.254:37861/ using document root <none> and password file <none>
I20260923 22:07:43.643951 3694 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:43.643991 3694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:43.644078 3694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:43.645044 3694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "a74380df5c364aa4b846eefad8bc2553"
format_stamp: "Formatted at 2026-09-23 22:07:43 on dist-test-slave-b5jz"
server_key: "400380cd900446310d825a682466f0c8"
server_key_iv: "69728ebd97b6d8da48b2a9232050af89"
server_key_version: "encryptionkey@0"
I20260923 22:07:43.645342 3694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "a74380df5c364aa4b846eefad8bc2553"
format_stamp: "Formatted at 2026-09-23 22:07:43 on dist-test-slave-b5jz"
server_key: "400380cd900446310d825a682466f0c8"
server_key_iv: "69728ebd97b6d8da48b2a9232050af89"
server_key_version: "encryptionkey@0"
I20260923 22:07:43.646582 3694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:07:43.647228 3709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:43.647378 3694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:43.647460 3694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "a74380df5c364aa4b846eefad8bc2553"
format_stamp: "Formatted at 2026-09-23 22:07:43 on dist-test-slave-b5jz"
server_key: "400380cd900446310d825a682466f0c8"
server_key_iv: "69728ebd97b6d8da48b2a9232050af89"
server_key_version: "encryptionkey@0"
I20260923 22:07:43.647518 3694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:43.677366 3694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:43.679793 3694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:43.679912 3694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:43.684321 3694 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:39595
I20260923 22:07:43.684410 3771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:39595 every 8 connection(s)
I20260923 22:07:43.684729 3694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:43.685487 3772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:43.687760 3772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553: Bootstrap starting.
I20260923 22:07:43.688323 3772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:43.688606 3772 log.cc:824] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:43.689378 3772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553: No bootstrap required, opened a new log
I20260923 22:07:43.691434 3772 raft_consensus.cc:348] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74380df5c364aa4b846eefad8bc2553" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39595 } }
I20260923 22:07:43.691664 3772 raft_consensus.cc:374] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:43.691731 3772 raft_consensus.cc:729] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a74380df5c364aa4b846eefad8bc2553, State: Initialized, Role: FOLLOWER
I20260923 22:07:43.691856 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 3694
I20260923 22:07:43.691895 3772 consensus_queue.cc:260] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74380df5c364aa4b846eefad8bc2553" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39595 } }
I20260923 22:07:43.691952 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:07:43.691979 3772 raft_consensus.cc:388] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:43.692014 3772 raft_consensus.cc:482] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:43.692060 3772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:43.692154 2003 external_mini_cluster.cc:1428] Setting key 6a29aae7ba2e6c1b27a870420e4cdae2
I20260923 22:07:43.692763 3772 raft_consensus.cc:504] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74380df5c364aa4b846eefad8bc2553" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39595 } }
I20260923 22:07:43.692879 3772 leader_election.cc:302] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a74380df5c364aa4b846eefad8bc2553; no voters:
I20260923 22:07:43.693055 3772 leader_election.cc:288] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:43.693086 3777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:43.693233 3777 raft_consensus.cc:686] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [term 1 LEADER]: Becoming Leader. State: Replica: a74380df5c364aa4b846eefad8bc2553, State: Running, Role: LEADER
I20260923 22:07:43.693284 3772 sys_catalog.cc:564] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:43.693351 3777 consensus_queue.cc:237] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74380df5c364aa4b846eefad8bc2553" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39595 } }
I20260923 22:07:43.693710 3778 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a74380df5c364aa4b846eefad8bc2553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74380df5c364aa4b846eefad8bc2553" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39595 } } }
I20260923 22:07:43.693838 3778 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:43.693710 3779 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a74380df5c364aa4b846eefad8bc2553. Latest consensus state: current_term: 1 leader_uuid: "a74380df5c364aa4b846eefad8bc2553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74380df5c364aa4b846eefad8bc2553" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39595 } } }
I20260923 22:07:43.693986 3779 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:43.694111 3784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:43.694752 3784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:43.696292 3784 catalog_manager.cc:1347] Generated new cluster ID: ee08fff61dec45d99d97a49cc091aa1a
I20260923 22:07:43.696648 3784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:07:43 dist-test-slave-b5jz krb5kdc[3681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:43.704502 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:43.692747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.1.244.254:39595)
0923 22:07:43.692984 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:43.692995 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:43.693032 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:07:43.693165 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:43.693173 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:43.693229 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:43.694130 (+ 901us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:43.694292 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.695890 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.697031 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.697667 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.697778 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:43.700404 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:43.700408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:43.700408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:43.700420 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:43.702859 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:43.703244 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:43.703247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:43.703248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:43.703368 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:43.703585 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:43.703586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:43.703586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:43.703893 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:43.704063 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:43.704145 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:07:43.704292 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
I20260923 22:07:43.705293 3784 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:43.705776 3784 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:43.708645 3784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553: Generated new TSK 0
I20260923 22:07:43.708825 3784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:07:43.738624 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39595
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:43.817541 3798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:43.817727 3798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:43.817750 3798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:43.818892 3798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:43.818939 3798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:43.818961 3798 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:07:43.819013 3798 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:07:43.819048 3798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:07:43.820500 3798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:39595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:43.820741 3798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:43.820976 3798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:43.825832 3806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:43.825843 3803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:43.825850 3804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:43.826153 3798 server_base.cc:1034] running on GCE node
I20260923 22:07:43.826375 3798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:43.826607 3798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:43.827796 3798 hybrid_clock.cc:648] HybridClock initialized: now 1790201263827758 us; error 36 us; skew 500 ppm
Sep 23 22:07:43 dist-test-slave-b5jz krb5kdc[3681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201263, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:43.830296 3798 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:07:43.830734 3798 webserver.cc:460] Webserver started at http://127.1.244.193:37661/ using document root <none> and password file <none>
I20260923 22:07:43.830958 3798 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:43.831000 3798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:43.831149 3798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:43.832191 3798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "809052d50ec74d1b9924d85c322bf3c8"
format_stamp: "Formatted at 2026-09-23 22:07:43 on dist-test-slave-b5jz"
server_key: "527771f7a264e22f2c6bea6950e413f2"
server_key_iv: "d10d61377e65a576120af26c100c5f5e"
server_key_version: "encryptionkey@0"
I20260923 22:07:43.832530 3798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "809052d50ec74d1b9924d85c322bf3c8"
format_stamp: "Formatted at 2026-09-23 22:07:43 on dist-test-slave-b5jz"
server_key: "527771f7a264e22f2c6bea6950e413f2"
server_key_iv: "d10d61377e65a576120af26c100c5f5e"
server_key_version: "encryptionkey@0"
I20260923 22:07:43.833849 3798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:43.834546 3813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:43.834721 3798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:43.834815 3798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "809052d50ec74d1b9924d85c322bf3c8"
format_stamp: "Formatted at 2026-09-23 22:07:43 on dist-test-slave-b5jz"
server_key: "527771f7a264e22f2c6bea6950e413f2"
server_key_iv: "d10d61377e65a576120af26c100c5f5e"
server_key_version: "encryptionkey@0"
I20260923 22:07:43.834892 3798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:43.848241 3798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:43.850459 3798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:43.850615 3798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:43.851176 3798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:43.851225 3798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:43.851269 3798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:43.851297 3798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:07:43 dist-test-slave-b5jz krb5kdc[3681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201263, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:43.858381 3798 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:44087
I20260923 22:07:43.858459 3928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:44087 every 8 connection(s)
I20260923 22:07:43.858785 3798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:07:43.860598 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:43.851577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53149 (local address 127.1.244.254:39595)
0923 22:07:43.851626 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:43.851629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:43.851858 (+ 229us) server_negotiation.cc:407] Connection header received
0923 22:07:43.852273 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:43.852275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:43.852292 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:43.852372 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:43.852932 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.853784 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.854826 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.855173 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.855255 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:43.857784 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:43.857786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:43.857786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:43.857795 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:43.859453 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:43.859855 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:43.859856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:43.859856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:43.859908 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:43.860128 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:43.860128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:43.860129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:43.860229 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:43.860276 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:43.860476 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 22:07:43.860537 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:07:43.860749 3822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:43.851538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39595 (local address 127.1.244.193:53149)
0923 22:07:43.851724 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:43.851762 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:43.852181 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:43.852420 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:43.852423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:43.852642 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:43.852812 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:43.852816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.853909 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.853910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:43.854720 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:43.854722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.855271 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.855272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:43.855404 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:43.855904 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:43.855913 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:43.857673 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:43.859519 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:43.859523 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:43.859524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:43.859788 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:43.859972 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:43.859973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:43.859973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:43.860067 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:43.860280 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:43.860284 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:43.860385 (+ 101us) client_negotiation.cc:769] Sending connection context
0923 22:07:43.860456 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:07:43.860574 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260923 22:07:43.861223 3929 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39595
I20260923 22:07:43.861303 3929 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:43.861425 3929 heartbeater.cc:511] Master 127.1.244.254:39595 requested a full tablet report, sending...
I20260923 22:07:43.861937 3725 ts_manager.cc:194] Registered new tserver with Master: 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087)
I20260923 22:07:43.862669 3725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:53149
I20260923 22:07:43.863277 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 3798
I20260923 22:07:43.863373 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:43.863539 2003 external_mini_cluster.cc:1428] Setting key 785d5bdd884ec8050641c0437ace39d8
I20260923 22:07:43.867781 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:43.862239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:38977 (local address 127.1.244.254:39595)
0923 22:07:43.862288 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:43.862290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:43.862435 (+ 145us) server_negotiation.cc:407] Connection header received
0923 22:07:43.862479 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:43.862480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:43.862490 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:43.862533 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:43.862836 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.863512 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.864277 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.864570 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.864640 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:43.865522 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:43.865524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:43.865524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:43.865530 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:43.866714 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:43.867089 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:43.867090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:43.867091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:43.867142 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:43.867337 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:43.867338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:43.867338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:43.867427 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:43.867476 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:43.867667 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 22:07:43.867726 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:07:43.867892 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:43.862244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39595 (local address 127.1.244.193:38977)
0923 22:07:43.862359 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:43.862369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:43.862427 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:43.862545 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:43.862545 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:43.862712 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:43.862755 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:43.862756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.863670 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.863671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:43.864210 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:43.864212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:43.864642 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:43.864643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:43.864697 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:43.865141 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:43.865146 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:43.865401 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:43.866799 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:43.866800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:43.866801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:43.867015 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:43.867210 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:43.867211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:43.867212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:43.867279 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:43.867487 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:43.867488 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:43.867535 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 22:07:43.867635 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 22:07:43.867748 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:07:43.887110 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39595
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:43.968267 3938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:43.968468 3938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:43.968492 3938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:43.969655 3938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:43.969697 3938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:43.969717 3938 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:07:43.969769 3938 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:07:43.969795 3938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:07:43.971225 3938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:39595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:43.971462 3938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:43.971725 3938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:43.976658 3943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:43.976616 3946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:43.976609 3944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:43.976944 3938 server_base.cc:1034] running on GCE node
I20260923 22:07:43.977180 3938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:43.977469 3938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:43.978607 3938 hybrid_clock.cc:648] HybridClock initialized: now 1790201263978597 us; error 36 us; skew 500 ppm
Sep 23 22:07:43 dist-test-slave-b5jz krb5kdc[3681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201263, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:43.981148 3938 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:07:43.981580 3938 webserver.cc:460] Webserver started at http://127.1.244.194:36179/ using document root <none> and password file <none>
I20260923 22:07:43.981799 3938 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:43.981894 3938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:43.982043 3938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:43.983103 3938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "99a77b07a2e241d1821feb7b434cb90a"
format_stamp: "Formatted at 2026-09-23 22:07:43 on dist-test-slave-b5jz"
server_key: "757dddfc0203ba81918431a721525b95"
server_key_iv: "bc696fae9be245eed5d20b6e195f90a6"
server_key_version: "encryptionkey@0"
I20260923 22:07:43.983458 3938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "99a77b07a2e241d1821feb7b434cb90a"
format_stamp: "Formatted at 2026-09-23 22:07:43 on dist-test-slave-b5jz"
server_key: "757dddfc0203ba81918431a721525b95"
server_key_iv: "bc696fae9be245eed5d20b6e195f90a6"
server_key_version: "encryptionkey@0"
I20260923 22:07:43.984762 3938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:07:43.985474 3953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:43.985649 3938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:43.985718 3938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "99a77b07a2e241d1821feb7b434cb90a"
format_stamp: "Formatted at 2026-09-23 22:07:43 on dist-test-slave-b5jz"
server_key: "757dddfc0203ba81918431a721525b95"
server_key_iv: "bc696fae9be245eed5d20b6e195f90a6"
server_key_version: "encryptionkey@0"
I20260923 22:07:43.985809 3938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:43.997401 3938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:43.999164 3938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:43.999325 3938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:43.999915 3938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:43.999977 3938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:44.000010 3938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:44.000048 3938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:07:44 dist-test-slave-b5jz krb5kdc[3681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201263, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:44.007130 3938 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:36931
I20260923 22:07:44.007203 4068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:36931 every 8 connection(s)
I20260923 22:07:44.007529 3938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:07:44.009992 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.000375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:45471 (local address 127.1.244.254:39595)
0923 22:07:44.000418 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.000420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.000642 (+ 222us) server_negotiation.cc:407] Connection header received
0923 22:07:44.001233 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.001235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.001250 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.001312 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:44.002139 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.002948 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.004704 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.005020 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.005095 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.007359 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:44.007361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:44.007362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:44.007371 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:44.008813 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.009228 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.009228 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.009229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.009276 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.009471 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.009472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.009472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.009566 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:44.009613 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.009844 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.009909 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:07:44.010103 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.000362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39595 (local address 127.1.244.194:45471)
0923 22:07:44.000525 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.000560 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.001127 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.001380 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.001384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.001756 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:44.002045 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.002052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.003066 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.003068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.004584 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.004588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.005123 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.005124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.005226 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.005790 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:44.005799 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:44.007233 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:44.008918 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.008924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.008925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.009164 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.009329 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.009330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.009330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.009419 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.009617 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:44.009620 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:44.009720 (+ 100us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.009817 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.009927 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260923 22:07:44.010586 4069 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39595
I20260923 22:07:44.010665 4069 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:44.010849 4069 heartbeater.cc:511] Master 127.1.244.254:39595 requested a full tablet report, sending...
I20260923 22:07:44.011166 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 3938
I20260923 22:07:44.011214 3726 ts_manager.cc:194] Registered new tserver with Master: 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931)
I20260923 22:07:44.011262 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:44.011441 2003 external_mini_cluster.cc:1428] Setting key 5f57f7d6282990abbbae1b8d0b7871bf
I20260923 22:07:44.011668 3726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:45471
I20260923 22:07:44.017812 4075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.011662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39595 (local address 127.1.244.194:57705)
0923 22:07:44.012029 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.012040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.012116 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.012279 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.012280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.012480 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:44.012541 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.012542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.013166 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.013167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.013796 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.013798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.014259 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.014260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.014313 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.014702 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:44.014706 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:44.014946 (+ 240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:44.016829 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.016829 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.016830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.017011 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.017215 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.017216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.017216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.017280 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.017606 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:44.017607 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:44.017638 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.017687 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.017753 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":66,"threads_started":1}
I20260923 22:07:44.017889 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.011706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:57705 (local address 127.1.244.254:39595)
0923 22:07:44.011943 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.011945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.012092 (+ 147us) server_negotiation.cc:407] Connection header received
0923 22:07:44.012184 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.012185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.012194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.012222 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:44.012619 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.013071 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.013874 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.014183 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.014257 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.015053 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:44.015054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:44.015054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:44.015061 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:44.016733 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.017082 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.017083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.017083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.017145 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.017323 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.017324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.017325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.017526 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:44.017614 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.017714 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.017790 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222}
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:07:44.037685 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39595
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:44.116961 4078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:44.117156 4078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:44.117182 4078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:44.118258 4078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:44.118291 4078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:44.118311 4078 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:07:44.118322 4078 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:07:44.118338 4078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:07:44.119742 4078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:39595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:44.119920 4078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:44.120092 4078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:44.124668 4084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:44.124722 4086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:44.124678 4083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:44.124864 4078 server_base.cc:1034] running on GCE node
I20260923 22:07:44.125104 4078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:44.125381 4078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:44.126555 4078 hybrid_clock.cc:648] HybridClock initialized: now 1790201264126536 us; error 47 us; skew 500 ppm
Sep 23 22:07:44 dist-test-slave-b5jz krb5kdc[3681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201264, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:44.129053 4078 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:07:44.129489 4078 webserver.cc:460] Webserver started at http://127.1.244.195:37443/ using document root <none> and password file <none>
I20260923 22:07:44.129722 4078 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:44.129791 4078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:44.129947 4078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:44.130955 4078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "29739c0d28e64cad96219608f7317a6d"
format_stamp: "Formatted at 2026-09-23 22:07:44 on dist-test-slave-b5jz"
server_key: "2abfe099c44537177379d2607b3ae4a2"
server_key_iv: "3bdc885da8675c1e2e8c09833126f3c2"
server_key_version: "encryptionkey@0"
I20260923 22:07:44.131309 4078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "29739c0d28e64cad96219608f7317a6d"
format_stamp: "Formatted at 2026-09-23 22:07:44 on dist-test-slave-b5jz"
server_key: "2abfe099c44537177379d2607b3ae4a2"
server_key_iv: "3bdc885da8675c1e2e8c09833126f3c2"
server_key_version: "encryptionkey@0"
I20260923 22:07:44.132665 4078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:07:44.133363 4093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:44.133527 4078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:44.133625 4078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "29739c0d28e64cad96219608f7317a6d"
format_stamp: "Formatted at 2026-09-23 22:07:44 on dist-test-slave-b5jz"
server_key: "2abfe099c44537177379d2607b3ae4a2"
server_key_iv: "3bdc885da8675c1e2e8c09833126f3c2"
server_key_version: "encryptionkey@0"
I20260923 22:07:44.133708 4078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:44.150980 4078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:44.152830 4078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:44.153018 4078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:44.153513 4078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:44.153577 4078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:44.153614 4078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:44.153662 4078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:07:44 dist-test-slave-b5jz krb5kdc[3681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201264, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:44.160723 4078 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:43171
I20260923 22:07:44.160760 4208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:43171 every 8 connection(s)
I20260923 22:07:44.161139 4078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:07:44.162101 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 4078
I20260923 22:07:44.162209 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:44.162425 2003 external_mini_cluster.cc:1428] Setting key 0095cab3ee6f1d3d5953f84a5110ce88
I20260923 22:07:44.163520 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.153994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:58657 (local address 127.1.244.254:39595)
0923 22:07:44.154055 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.154057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.154289 (+ 232us) server_negotiation.cc:407] Connection header received
0923 22:07:44.154797 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.154799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.154810 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.154865 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:44.155742 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.156565 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.157640 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.157856 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.157943 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.160114 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:44.160116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:44.160116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:44.160125 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:44.161832 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.162321 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.162323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.162323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.162383 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.162722 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.162723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.162724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.162883 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:44.162971 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.163348 (+ 377us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.163445 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:07:44.163733 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.154006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39595 (local address 127.1.244.195:58657)
0923 22:07:44.154172 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.154205 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.154695 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.154944 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.154947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.155309 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:44.155612 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.155617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.156669 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.156670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.157544 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.157547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.157965 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.157966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.158088 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.158632 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:44.158641 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:44.160012 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:44.161900 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.161904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.161906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.162227 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.162476 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.162477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.162477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.162622 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.163018 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:44.163029 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:44.163187 (+ 158us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.163309 (+ 122us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.163476 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260923 22:07:44.164170 4209 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39595
I20260923 22:07:44.164254 4209 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:44.164428 4209 heartbeater.cc:511] Master 127.1.244.254:39595 requested a full tablet report, sending...
I20260923 22:07:44.164805 3726 ts_manager.cc:194] Registered new tserver with Master: 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171)
I20260923 22:07:44.165177 3726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:58657
I20260923 22:07:44.170063 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.165269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:42677 (local address 127.1.244.254:39595)
0923 22:07:44.165309 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.165310 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.165534 (+ 224us) server_negotiation.cc:407] Connection header received
0923 22:07:44.165595 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.165596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.165606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.165667 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:44.165943 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.166403 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.167079 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.167247 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.167294 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.168080 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:44.168081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:44.168081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:44.168087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:44.169184 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.169521 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.169522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.169522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.169576 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.169716 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.169717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.169717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.169800 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:44.169846 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.169960 (+ 114us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.170010 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:07:44.170082 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.165229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39595 (local address 127.1.244.195:42677)
0923 22:07:44.165454 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.165466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.165529 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.165669 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.165670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.165824 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:44.165878 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.165879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.166487 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.166488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.167008 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.167010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.167307 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.167308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.167351 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.167750 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:44.167753 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:44.168009 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:44.169261 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.169262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.169262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.169457 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.169615 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.169616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.169616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.169671 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.169855 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:44.169855 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:44.169892 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.169940 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.170012 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":150,"threads_started":1}
I20260923 22:07:44.173991 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:07:44 dist-test-slave-b5jz krb5kdc[3681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:07:44 dist-test-slave-b5jz krb5kdc[3681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:44.195887 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.188831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.1.244.254:39595)
0923 22:07:44.188885 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.188889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.188902 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:07:44.188964 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.188965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.188973 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.189024 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:44.189611 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.190078 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.190756 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.190921 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.190975 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.193339 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:44.193340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:44.193342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:44.193349 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:44.194750 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.195100 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.195101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.195102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.195158 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.195352 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.195353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.195353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.195460 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:44.195600 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.195714 (+ 114us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.195788 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:07:44.197443 3726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:07:44.198871 3726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:44.205210 4224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.200902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:43171 (local address 127.0.0.1:33318)
0923 22:07:44.201057 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.201106 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.201170 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.201545 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.201548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.201550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:44.201610 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.201612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.202848 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.202849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.204136 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.204137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.204988 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.204988 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.205071 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.205075 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.205123 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.205154 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":67,"thread_start_us":51,"threads_started":1}
I20260923 22:07:44.205221 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.201029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46704 (local address 127.1.244.194:36931)
0923 22:07:44.201316 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.201321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.201408 (+ 87us) server_negotiation.cc:407] Connection header received
0923 22:07:44.201494 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.201496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.201510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.201580 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:44.201828 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.202842 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.204033 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.204796 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.204882 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.204967 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.205087 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.205126 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":84,"threads_started":1}
I20260923 22:07:44.205197 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.200914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:36931 (local address 127.0.0.1:46704)
0923 22:07:44.201356 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.201364 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.201415 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.201640 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.201641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.201642 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:44.201748 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.201751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.202962 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.202963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.203949 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.203951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.204873 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.204873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.204968 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.204984 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.205060 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.205121 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":58,"threads_started":1}
I20260923 22:07:44.205407 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.201207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.1.244.195:43171)
0923 22:07:44.201360 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.201364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.201375 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:07:44.201415 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.201417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.201433 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.201538 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:44.201695 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.202769 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.204232 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.204889 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.204981 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.205054 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.205252 (+ 198us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.205302 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260923 22:07:44.206048 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7aea12434e4a5a8e897bc9e9e1e64b (DEFAULT_TABLE table=test-table [id=9d4a531528a44b4098615c447c19343c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:44.206094 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7aea12434e4a5a8e897bc9e9e1e64b (DEFAULT_TABLE table=test-table [id=9d4a531528a44b4098615c447c19343c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:44.206403 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7aea12434e4a5a8e897bc9e9e1e64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:44.206403 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7aea12434e4a5a8e897bc9e9e1e64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:44.206446 4223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.201098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44087 (local address 127.0.0.1:50508)
0923 22:07:44.201375 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.201386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.201435 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.201864 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.201865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.201867 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:44.201909 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.201911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.202740 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.202741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.204134 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.204137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.206226 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.206227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.206287 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.206291 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.206334 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.206371 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":64,"thread_start_us":48,"threads_started":1}
I20260923 22:07:44.206719 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.201221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50508 (local address 127.1.244.193:44087)
0923 22:07:44.201667 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.201671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.201681 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:07:44.201715 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.201717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.201731 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.201800 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:44.201982 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.202653 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.205095 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.206121 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.206312 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.206375 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.206401 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.206640 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":75,"threads_started":1}
I20260923 22:07:44.207253 3848 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7aea12434e4a5a8e897bc9e9e1e64b (DEFAULT_TABLE table=test-table [id=9d4a531528a44b4098615c447c19343c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:44.207597 3848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7aea12434e4a5a8e897bc9e9e1e64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:44.209671 4229 tablet_bootstrap.cc:492] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: Bootstrap starting.
I20260923 22:07:44.209993 4230 tablet_bootstrap.cc:492] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: Bootstrap starting.
I20260923 22:07:44.210590 4230 tablet_bootstrap.cc:654] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:44.210722 4231 tablet_bootstrap.cc:492] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: Bootstrap starting.
I20260923 22:07:44.210918 4230 log.cc:824] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:44.211048 4229 tablet_bootstrap.cc:654] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:44.211301 4231 tablet_bootstrap.cc:654] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:44.211362 4229 log.cc:824] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:44.211717 4231 log.cc:824] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:44.211890 4230 tablet_bootstrap.cc:492] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: No bootstrap required, opened a new log
I20260923 22:07:44.211982 4230 ts_tablet_manager.cc:1397] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:44.212105 4229 tablet_bootstrap.cc:492] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: No bootstrap required, opened a new log
I20260923 22:07:44.212199 4229 ts_tablet_manager.cc:1397] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260923 22:07:44.212450 4231 tablet_bootstrap.cc:492] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: No bootstrap required, opened a new log
I20260923 22:07:44.212513 4231 ts_tablet_manager.cc:1397] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:44.213454 4230 raft_consensus.cc:348] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } }
I20260923 22:07:44.213601 4230 raft_consensus.cc:374] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:44.213644 4230 raft_consensus.cc:729] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a77b07a2e241d1821feb7b434cb90a, State: Initialized, Role: FOLLOWER
I20260923 22:07:44.213747 4230 consensus_queue.cc:260] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } }
I20260923 22:07:44.214035 4069 heartbeater.cc:503] Master 127.1.244.254:39595 was elected leader, sending a full tablet report...
I20260923 22:07:44.214143 4229 raft_consensus.cc:348] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } }
I20260923 22:07:44.214308 4229 raft_consensus.cc:374] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:44.214324 4230 ts_tablet_manager.cc:1428] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:44.214345 4229 raft_consensus.cc:729] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29739c0d28e64cad96219608f7317a6d, State: Initialized, Role: FOLLOWER
I20260923 22:07:44.214454 4229 consensus_queue.cc:260] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } }
I20260923 22:07:44.214418 4231 raft_consensus.cc:348] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } }
I20260923 22:07:44.214637 4231 raft_consensus.cc:374] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:44.214681 4231 raft_consensus.cc:729] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809052d50ec74d1b9924d85c322bf3c8, State: Initialized, Role: FOLLOWER
I20260923 22:07:44.214732 4229 ts_tablet_manager.cc:1428] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:07:44.214834 4209 heartbeater.cc:503] Master 127.1.244.254:39595 was elected leader, sending a full tablet report...
I20260923 22:07:44.214802 4231 consensus_queue.cc:260] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } }
I20260923 22:07:44.215036 4231 ts_tablet_manager.cc:1428] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:07:44.215210 3929 heartbeater.cc:503] Master 127.1.244.254:39595 was elected leader, sending a full tablet report...
W20260923 22:07:44.258116 4070 tablet.cc:2369] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:07:44.359486 3930 tablet.cc:2369] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:07:44.371126 4236 raft_consensus.cc:482] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:44.371239 4236 raft_consensus.cc:504] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } }
I20260923 22:07:44.371609 4236 leader_election.cc:288] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931), 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087)
I20260923 22:07:44.374655 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.371784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:36931 (local address 127.1.244.195:41087)
0923 22:07:44.371830 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.371844 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.371894 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.372048 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.372049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.372051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:44.372123 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.372125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.372893 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.372894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.373709 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.373712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.374357 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.374358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.374451 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.374455 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.374527 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.374572 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 22:07:44.374667 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.371847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:41087 (local address 127.1.244.194:36931)
0923 22:07:44.371904 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.371906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.371918 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:07:44.371952 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.371953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.371964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.372027 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:44.372198 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.372808 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.373791 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.374269 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.374324 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.374331 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.374537 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.374595 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:07:44.374894 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.371905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:43217 (local address 127.1.244.193:44087)
0923 22:07:44.371954 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.371956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.372208 (+ 252us) server_negotiation.cc:407] Connection header received
0923 22:07:44.372286 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.372287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.372296 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.372334 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:44.372478 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.372969 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.374071 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.374544 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.374609 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.374617 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.374789 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.374835 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:07:44.374909 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.371785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44087 (local address 127.1.244.195:43217)
0923 22:07:44.372134 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.372143 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.372229 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.372353 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.372354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.372355 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:44.372417 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.372419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.373067 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.373069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.373969 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.373972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.374627 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.374628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.374715 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.374720 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.374781 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.374818 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":90,"thread_start_us":142,"threads_started":1}
I20260923 22:07:44.375043 4013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7aea12434e4a5a8e897bc9e9e1e64b" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a77b07a2e241d1821feb7b434cb90a" is_pre_election: true
I20260923 22:07:44.375208 4013 raft_consensus.cc:2454] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 0.
I20260923 22:07:44.375226 3883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7aea12434e4a5a8e897bc9e9e1e64b" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8" is_pre_election: true
I20260923 22:07:44.375391 3883 raft_consensus.cc:2454] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 0.
I20260923 22:07:44.375425 4095 leader_election.cc:302] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 99a77b07a2e241d1821feb7b434cb90a; no voters:
I20260923 22:07:44.375602 4236 raft_consensus.cc:2781] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:44.375658 4236 raft_consensus.cc:482] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:44.375684 4236 raft_consensus.cc:3037] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:44.376397 4236 raft_consensus.cc:504] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } }
I20260923 22:07:44.376539 4236 leader_election.cc:288] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 election: Requested vote from peers 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931), 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087)
I20260923 22:07:44.376708 3883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7aea12434e4a5a8e897bc9e9e1e64b" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8"
I20260923 22:07:44.376722 4013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7aea12434e4a5a8e897bc9e9e1e64b" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a77b07a2e241d1821feb7b434cb90a"
I20260923 22:07:44.376794 3883 raft_consensus.cc:3037] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:44.376806 4013 raft_consensus.cc:3037] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:44.377656 3883 raft_consensus.cc:2454] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 1.
I20260923 22:07:44.377656 4013 raft_consensus.cc:2454] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 1.
I20260923 22:07:44.377853 4095 leader_election.cc:302] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 99a77b07a2e241d1821feb7b434cb90a; no voters:
I20260923 22:07:44.377961 4236 raft_consensus.cc:2781] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:44.378108 4236 raft_consensus.cc:686] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 1 LEADER]: Becoming Leader. State: Replica: 29739c0d28e64cad96219608f7317a6d, State: Running, Role: LEADER
I20260923 22:07:44.378225 4236 consensus_queue.cc:237] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } }
I20260923 22:07:44.379002 3726 catalog_manager.cc:5495] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 29739c0d28e64cad96219608f7317a6d (127.1.244.195). New cstate: current_term: 1 leader_uuid: "29739c0d28e64cad96219608f7317a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } health_report { overall_health: UNKNOWN } } }
W20260923 22:07:44.411857 4210 tablet.cc:2369] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 22:07:44 dist-test-slave-b5jz krb5kdc[3681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201263, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:44.418644 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.412331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54230 (local address 127.1.244.254:39595)
0923 22:07:44.412392 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.412398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.412455 (+ 57us) server_negotiation.cc:407] Connection header received
0923 22:07:44.412501 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.412501 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.412510 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.412565 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:44.412985 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.413468 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.414137 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.414314 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.414370 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.416320 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:44.416321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:44.416321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:44.416327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:44.417751 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.418063 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.418064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.418064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.418113 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:44.418271 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:44.418271 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:44.418271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:44.418362 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:44.418412 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.418521 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.418583 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:07:44.418713 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.412242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39595 (local address 127.0.0.1:54230)
0923 22:07:44.412383 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.412394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.412459 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.412581 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.412582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.412859 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:44.412905 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.412907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.413535 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.413535 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.414050 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.414051 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.414379 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.414380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.414419 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.414898 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:44.414903 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:44.416244 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:44.417816 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.417822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.417823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.418001 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.418166 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:44.418167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:44.418167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:44.418219 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:44.418412 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:44.418417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:44.418453 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.418506 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.418629 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":43,"thread_start_us":81,"threads_started":1}
I20260923 22:07:44.420548 3726 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:54230:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:07:44.420781 3726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:44.422197 4003 tablet_service.cc:1467] Processing CreateTablet for tablet b9b4fb40de0d477d955400093bc5ea32 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=89ede345b3ba41b487cd2e65cd6cd06c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:07:44.422183 3848 tablet_service.cc:1467] Processing CreateTablet for tablet b9b4fb40de0d477d955400093bc5ea32 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=89ede345b3ba41b487cd2e65cd6cd06c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:07:44.422228 4143 tablet_service.cc:1467] Processing CreateTablet for tablet b9b4fb40de0d477d955400093bc5ea32 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=89ede345b3ba41b487cd2e65cd6cd06c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:07:44.422326 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9b4fb40de0d477d955400093bc5ea32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:44.422338 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9b4fb40de0d477d955400093bc5ea32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:44.422338 3848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9b4fb40de0d477d955400093bc5ea32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:44.423629 4230 tablet_bootstrap.cc:492] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: Bootstrap starting.
I20260923 22:07:44.423933 4231 tablet_bootstrap.cc:492] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: Bootstrap starting.
I20260923 22:07:44.424335 4231 tablet_bootstrap.cc:654] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:44.424335 4230 tablet_bootstrap.cc:654] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:44.423841 4229 tablet_bootstrap.cc:492] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: Bootstrap starting.
I20260923 22:07:44.425129 4229 tablet_bootstrap.cc:654] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:44.425148 4230 tablet_bootstrap.cc:492] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: No bootstrap required, opened a new log
I20260923 22:07:44.425189 4230 ts_tablet_manager.cc:1397] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:44.425346 4230 raft_consensus.cc:348] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.425426 4230 raft_consensus.cc:374] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:44.425450 4230 raft_consensus.cc:729] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a77b07a2e241d1821feb7b434cb90a, State: Initialized, Role: FOLLOWER
I20260923 22:07:44.425494 4230 consensus_queue.cc:260] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.425606 4231 tablet_bootstrap.cc:492] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: No bootstrap required, opened a new log
I20260923 22:07:44.425613 4230 ts_tablet_manager.cc:1428] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:44.425654 4231 ts_tablet_manager.cc:1397] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:44.425765 4235 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.425822 4229 tablet_bootstrap.cc:492] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: No bootstrap required, opened a new log
I20260923 22:07:44.425803 4231 raft_consensus.cc:348] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.425832 4235 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:44.425863 4231 raft_consensus.cc:374] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:44.425885 4231 raft_consensus.cc:729] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809052d50ec74d1b9924d85c322bf3c8, State: Initialized, Role: FOLLOWER
I20260923 22:07:44.425866 4229 ts_tablet_manager.cc:1397] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:07:44.425932 4231 consensus_queue.cc:260] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.426023 4231 ts_tablet_manager.cc:1428] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:44.426069 4229 raft_consensus.cc:348] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.426072 4237 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.426146 4229 raft_consensus.cc:374] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:44.426151 4237 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:44.426170 4229 raft_consensus.cc:729] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29739c0d28e64cad96219608f7317a6d, State: Initialized, Role: FOLLOWER
I20260923 22:07:44.426226 4229 consensus_queue.cc:260] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.426326 4229 ts_tablet_manager.cc:1428] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:44.426383 4236 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.426470 4236 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:44.563676 4235 raft_consensus.cc:482] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:44.563794 4235 raft_consensus.cc:504] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.564160 4235 leader_election.cc:288] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087), 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171)
I20260923 22:07:44.568169 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.564629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:43253 (local address 127.1.244.195:43171)
0923 22:07:44.564677 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.564679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.564732 (+ 53us) server_negotiation.cc:407] Connection header received
0923 22:07:44.564784 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.564785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.564792 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.564845 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:44.564992 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.565656 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.567227 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.567819 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.567879 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.567895 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.568046 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.568087 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:07:44.568190 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.564543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:43171 (local address 127.1.244.194:43253)
0923 22:07:44.564678 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.564687 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.564737 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.564862 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.564863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.564865 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:44.564926 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.564927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.565753 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.565754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.567124 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.567126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.567906 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.567907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.567965 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.567968 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.568018 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.568084 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":123,"thread_start_us":58,"threads_started":1}
I20260923 22:07:44.568400 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.564422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:49183 (local address 127.1.244.193:44087)
0923 22:07:44.564479 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.564481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.564550 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:07:44.564671 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.564672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.564683 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.564734 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:44.564904 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.565648 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.567228 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.567798 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.567878 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.567887 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.568277 (+ 390us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.568320 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:07:44.568493 4163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b4fb40de0d477d955400093bc5ea32" candidate_uuid: "99a77b07a2e241d1821feb7b434cb90a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29739c0d28e64cad96219608f7317a6d" is_pre_election: true
I20260923 22:07:44.568190 4250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.564355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44087 (local address 127.1.244.194:49183)
0923 22:07:44.564483 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.564494 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.564558 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.564741 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.564742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.564744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:44.564804 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.564806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.565754 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.565755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.567123 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.567126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.567885 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.567887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.567960 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.567964 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.568024 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.568078 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":66,"thread_start_us":68,"threads_started":1}
I20260923 22:07:44.568615 4163 raft_consensus.cc:2454] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a77b07a2e241d1821feb7b434cb90a in term 0.
I20260923 22:07:44.568823 3956 leader_election.cc:302] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 99a77b07a2e241d1821feb7b434cb90a; no voters:
I20260923 22:07:44.568849 3883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b4fb40de0d477d955400093bc5ea32" candidate_uuid: "99a77b07a2e241d1821feb7b434cb90a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8" is_pre_election: true
I20260923 22:07:44.568926 3883 raft_consensus.cc:2454] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a77b07a2e241d1821feb7b434cb90a in term 0.
I20260923 22:07:44.569046 4235 raft_consensus.cc:2781] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:44.569109 4235 raft_consensus.cc:482] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:44.569140 4235 raft_consensus.cc:3037] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:44.569818 4235 raft_consensus.cc:504] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.569974 4235 leader_election.cc:288] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [CANDIDATE]: Term 1 election: Requested vote from peers 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087), 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171)
I20260923 22:07:44.570139 3883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b4fb40de0d477d955400093bc5ea32" candidate_uuid: "99a77b07a2e241d1821feb7b434cb90a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8"
I20260923 22:07:44.570142 4163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b4fb40de0d477d955400093bc5ea32" candidate_uuid: "99a77b07a2e241d1821feb7b434cb90a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29739c0d28e64cad96219608f7317a6d"
I20260923 22:07:44.570205 3883 raft_consensus.cc:3037] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:44.570228 4163 raft_consensus.cc:3037] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:44.570788 3883 raft_consensus.cc:2454] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a77b07a2e241d1821feb7b434cb90a in term 1.
I20260923 22:07:44.570968 3956 leader_election.cc:302] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809052d50ec74d1b9924d85c322bf3c8, 99a77b07a2e241d1821feb7b434cb90a; no voters:
I20260923 22:07:44.571071 4235 raft_consensus.cc:2781] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:44.570952 4163 raft_consensus.cc:2454] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a77b07a2e241d1821feb7b434cb90a in term 1.
I20260923 22:07:44.571211 4235 raft_consensus.cc:686] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [term 1 LEADER]: Becoming Leader. State: Replica: 99a77b07a2e241d1821feb7b434cb90a, State: Running, Role: LEADER
I20260923 22:07:44.571302 4235 consensus_queue.cc:237] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.571743 4252 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: TxnStatusTablet state changed. Reason: New leader 99a77b07a2e241d1821feb7b434cb90a. Latest consensus state: current_term: 1 leader_uuid: "99a77b07a2e241d1821feb7b434cb90a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.571835 4252 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:07:44.572006 4254 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:07:44.572127 4254 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:07:44.572324 3725 catalog_manager.cc:5495] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a reported cstate change: term changed from 0 to 1, leader changed from <none> to 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194). New cstate: current_term: 1 leader_uuid: "99a77b07a2e241d1821feb7b434cb90a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } health_report { overall_health: UNKNOWN } } }
I20260923 22:07:44.608767 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.606183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:36931 (local address 127.0.0.1:46714)
0923 22:07:44.606246 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.606257 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.606318 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.606508 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.606509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.606510 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:07:44.606560 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.606569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.607193 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.607194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.607857 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.607859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.608226 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.608227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.608293 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.608314 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.608592 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.608601 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.608651 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.608699 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260923 22:07:44.608754 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.606272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.1.244.194:36931)
0923 22:07:44.606336 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.606338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.606364 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:07:44.606396 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.606397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.606417 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.606502 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:44.606634 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.607106 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.607937 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.608139 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.608227 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.608419 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.608522 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.608617 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.608657 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.608678 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:07:44.610654 3883 raft_consensus.cc:1261] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Refusing update from remote peer 99a77b07a2e241d1821feb7b434cb90a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:44.610698 4163 raft_consensus.cc:1261] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Refusing update from remote peer 99a77b07a2e241d1821feb7b434cb90a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:44.610967 4235 consensus_queue.cc:1035] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [LEADER]: Connected to new peer: Peer: permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:44.611109 4252 consensus_queue.cc:1035] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a [LEADER]: Connected to new peer: Peer: permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:44.612251 4237 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: TxnStatusTablet state changed. Reason: New leader 99a77b07a2e241d1821feb7b434cb90a. Latest consensus state: current_term: 1 leader_uuid: "99a77b07a2e241d1821feb7b434cb90a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.612277 4236 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: TxnStatusTablet state changed. Reason: New leader 99a77b07a2e241d1821feb7b434cb90a. Latest consensus state: current_term: 1 leader_uuid: "99a77b07a2e241d1821feb7b434cb90a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.612344 4237 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:44.612355 4236 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:44.613534 4235 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99a77b07a2e241d1821feb7b434cb90a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.613642 4235 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:07:44.613656 4236 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99a77b07a2e241d1821feb7b434cb90a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.613693 4235 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99a77b07a2e241d1821feb7b434cb90a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.613727 4236 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 29739c0d28e64cad96219608f7317a6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:44.613739 4257 mvcc.cc:205] Tried to move back new op lower bound from 7332664379842449408 to 7332664379684589568. Current Snapshot: MvccSnapshot[applied={T|T < 7332664379842449408}]
I20260923 22:07:44.613824 4259 mvcc.cc:205] Tried to move back new op lower bound from 7332664379842449408 to 7332664379684589568. Current Snapshot: MvccSnapshot[applied={T|T < 7332664379842449408}]
I20260923 22:07:44.614590 4237 tablet_replica.cc:439] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99a77b07a2e241d1821feb7b434cb90a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } } }
I20260923 22:07:44.614624 4258 mvcc.cc:205] Tried to move back new op lower bound from 7332664379842449408 to 7332664379684589568. Current Snapshot: MvccSnapshot[applied={T|T < 7332664379842449408}]
I20260923 22:07:44.614687 4237 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 809052d50ec74d1b9924d85c322bf3c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:07:44.613745 4235 tablet_replica.cc:442] T b9b4fb40de0d477d955400093bc5ea32 P 99a77b07a2e241d1821feb7b434cb90a: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:07:44.618779 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.616423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33326 (local address 127.1.244.195:43171)
0923 22:07:44.616468 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.616470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.616480 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:07:44.616547 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.616548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.616572 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.616627 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:44.616790 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.617255 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.618008 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.618212 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.618284 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.618459 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.618551 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.618642 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.618682 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.618703 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:07:44.623631 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.621065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:59429 (local address 127.1.244.194:36931)
0923 22:07:44.621123 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.621125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.621175 (+ 50us) server_negotiation.cc:407] Connection header received
0923 22:07:44.621247 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.621247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.621255 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.621299 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:44.621421 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.621840 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.622890 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.623119 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.623172 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.623367 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.623391 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.623429 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.623509 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.623523 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:07:44.623636 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.621002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:36931 (local address 127.1.244.195:59429)
0923 22:07:44.621126 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.621137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.621189 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.621302 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.621303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.621304 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:07:44.621360 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.621362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.621918 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.621919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.622797 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.622799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.623193 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.623193 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.623260 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.623274 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.623451 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.623453 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.623505 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.623565 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260923 22:07:44.625747 3883 raft_consensus.cc:1261] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Refusing update from remote peer 29739c0d28e64cad96219608f7317a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:44.625890 4013 raft_consensus.cc:1261] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Refusing update from remote peer 29739c0d28e64cad96219608f7317a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:44.625967 4239 consensus_queue.cc:1035] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:44.626123 4236 consensus_queue.cc:1035] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:44.634181 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.631268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:34005 (local address 127.1.244.195:43171)
0923 22:07:44.631312 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.631314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.631413 (+ 99us) server_negotiation.cc:407] Connection header received
0923 22:07:44.631498 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.631498 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.631508 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.631636 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:44.631736 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.632349 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.633527 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.633723 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.633803 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.633968 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.633998 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.634045 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.634100 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.634117 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:07:44.634213 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.631213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:43171 (local address 127.1.244.194:34005)
0923 22:07:44.631349 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.631359 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.631426 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.631640 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.631641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.631642 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:07:44.631684 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.631685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.632432 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.632433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.633433 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.633435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.633805 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.633805 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.633858 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.633871 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.634044 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.634045 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.634095 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.634136 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260923 22:07:44.642583 3725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:44.642689 3725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:44.643363 3725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553: Sending DeleteTablet for 3 replicas of tablet 8e7aea12434e4a5a8e897bc9e9e1e64b
I20260923 22:07:44.643644 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7aea12434e4a5a8e897bc9e9e1e64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:44 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33318
I20260923 22:07:44.643705 3848 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7aea12434e4a5a8e897bc9e9e1e64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:44 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:50508
I20260923 22:07:44.644017 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7aea12434e4a5a8e897bc9e9e1e64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:44 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46704
I20260923 22:07:44.644181 3725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:07:44.644280 4277 tablet_replica.cc:330] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: stopping tablet replica
W20260923 22:07:44.644321 3725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:44.644368 4277 raft_consensus.cc:2229] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:44.644413 4277 raft_consensus.cc:2258] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:44.645428 3848 tablet_service.cc:1467] Processing CreateTablet for tablet 953d2c47ce1840cbb86a5a90cc070a68 (DEFAULT_TABLE table=test-table [id=af478942dcca4ca284907361331a5b63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:44.645469 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 953d2c47ce1840cbb86a5a90cc070a68 (DEFAULT_TABLE table=test-table [id=af478942dcca4ca284907361331a5b63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:44.645555 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 953d2c47ce1840cbb86a5a90cc070a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:44.645555 3848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 953d2c47ce1840cbb86a5a90cc070a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:44.646874 4231 tablet_bootstrap.cc:492] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: Bootstrap starting.
I20260923 22:07:44.646874 4229 tablet_bootstrap.cc:492] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: Bootstrap starting.
I20260923 22:07:44.647281 4231 tablet_bootstrap.cc:654] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:44.647281 4229 tablet_bootstrap.cc:654] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:44.647878 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 953d2c47ce1840cbb86a5a90cc070a68 (DEFAULT_TABLE table=test-table [id=af478942dcca4ca284907361331a5b63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:44.647989 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 953d2c47ce1840cbb86a5a90cc070a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:44.648893 4230 tablet_bootstrap.cc:492] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: Bootstrap starting.
I20260923 22:07:44.649312 4230 tablet_bootstrap.cc:654] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:44.649749 4277 ts_tablet_manager.cc:1905] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:44.650183 4230 tablet_bootstrap.cc:492] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: No bootstrap required, opened a new log
I20260923 22:07:44.650226 4230 ts_tablet_manager.cc:1397] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:44.650350 4230 raft_consensus.cc:348] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.650429 4230 raft_consensus.cc:374] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:44.650439 4276 tablet_replica.cc:330] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: stopping tablet replica
I20260923 22:07:44.650452 4230 raft_consensus.cc:729] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a77b07a2e241d1821feb7b434cb90a, State: Initialized, Role: FOLLOWER
I20260923 22:07:44.650494 4230 consensus_queue.cc:260] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.650533 4276 raft_consensus.cc:2229] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:44.650583 4276 raft_consensus.cc:2258] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:44.650583 4230 ts_tablet_manager.cc:1428] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:44.652063 4231 tablet_bootstrap.cc:492] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: No bootstrap required, opened a new log
I20260923 22:07:44.652072 4229 tablet_bootstrap.cc:492] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: No bootstrap required, opened a new log
I20260923 22:07:44.652113 4231 ts_tablet_manager.cc:1397] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260923 22:07:44.652113 4229 ts_tablet_manager.cc:1397] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260923 22:07:44.652253 4231 raft_consensus.cc:348] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.652264 4229 raft_consensus.cc:348] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.652328 4231 raft_consensus.cc:374] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:44.652330 4229 raft_consensus.cc:374] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:44.652351 4231 raft_consensus.cc:729] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809052d50ec74d1b9924d85c322bf3c8, State: Initialized, Role: FOLLOWER
I20260923 22:07:44.652352 4229 raft_consensus.cc:729] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29739c0d28e64cad96219608f7317a6d, State: Initialized, Role: FOLLOWER
I20260923 22:07:44.652390 4231 consensus_queue.cc:260] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.652391 4229 consensus_queue.cc:260] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.652477 4231 ts_tablet_manager.cc:1428] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:44.652482 4229 ts_tablet_manager.cc:1428] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:44.652758 4276 ts_tablet_manager.cc:1905] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:44.652949 4277 ts_tablet_manager.cc:1918] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:07:44.653120 4277 log.cc:1196] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/8e7aea12434e4a5a8e897bc9e9e1e64b
I20260923 22:07:44.653534 4275 tablet_replica.cc:330] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: stopping tablet replica
I20260923 22:07:44.653640 4275 raft_consensus.cc:2229] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:07:44.653736 4275 raft_consensus.cc:2258] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:44.654011 4277 ts_tablet_manager.cc:1939] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 99a77b07a2e241d1821feb7b434cb90a: Deleting consensus metadata
I20260923 22:07:44.654415 3711 catalog_manager.cc:4841] TS 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931): tablet 8e7aea12434e4a5a8e897bc9e9e1e64b (table test-table [id=9d4a531528a44b4098615c447c19343c]) successfully deleted
I20260923 22:07:44.654541 4276 ts_tablet_manager.cc:1918] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:07:44.654616 4276 log.cc:1196] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/8e7aea12434e4a5a8e897bc9e9e1e64b
I20260923 22:07:44.654649 4275 ts_tablet_manager.cc:1905] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:44.654896 4276 ts_tablet_manager.cc:1939] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 809052d50ec74d1b9924d85c322bf3c8: Deleting consensus metadata
I20260923 22:07:44.655309 3712 catalog_manager.cc:4841] TS 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087): tablet 8e7aea12434e4a5a8e897bc9e9e1e64b (table test-table [id=9d4a531528a44b4098615c447c19343c]) successfully deleted
I20260923 22:07:44.656006 4275 ts_tablet_manager.cc:1918] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:07:44.656239 4275 log.cc:1196] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/8e7aea12434e4a5a8e897bc9e9e1e64b
I20260923 22:07:44.656525 4275 ts_tablet_manager.cc:1939] T 8e7aea12434e4a5a8e897bc9e9e1e64b P 29739c0d28e64cad96219608f7317a6d: Deleting consensus metadata
I20260923 22:07:44.656900 3712 catalog_manager.cc:4841] TS 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171): tablet 8e7aea12434e4a5a8e897bc9e9e1e64b (table test-table [id=9d4a531528a44b4098615c447c19343c]) successfully deleted
I20260923 22:07:44.839895 4237 raft_consensus.cc:482] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:44.839993 4237 raft_consensus.cc:504] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.840363 4237 leader_election.cc:288] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931), 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171)
I20260923 22:07:44.843003 4252 raft_consensus.cc:482] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:44.843108 4252 raft_consensus.cc:504] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.843251 4252 leader_election.cc:288] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087), 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171)
I20260923 22:07:44.843417 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.840620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:60401 (local address 127.1.244.194:36931)
0923 22:07:44.840670 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.840672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.840733 (+ 61us) server_negotiation.cc:407] Connection header received
0923 22:07:44.840804 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.840805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.840814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.840886 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:44.841057 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.841523 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.842612 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.843040 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.843124 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.843132 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.843313 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.843347 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:07:44.843432 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.840605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:37037 (local address 127.1.244.195:43171)
0923 22:07:44.840659 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.840661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.840756 (+ 95us) server_negotiation.cc:407] Connection header received
0923 22:07:44.840825 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.840827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.840837 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.840910 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:44.841063 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.841520 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.842614 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.843049 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.843117 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.843127 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.843322 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.843364 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:07:44.843454 4163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953d2c47ce1840cbb86a5a90cc070a68" candidate_uuid: "99a77b07a2e241d1821feb7b434cb90a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29739c0d28e64cad96219608f7317a6d" is_pre_election: true
I20260923 22:07:44.843452 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953d2c47ce1840cbb86a5a90cc070a68" candidate_uuid: "99a77b07a2e241d1821feb7b434cb90a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8" is_pre_election: true
I20260923 22:07:44.843531 3882 raft_consensus.cc:2454] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a77b07a2e241d1821feb7b434cb90a in term 0.
I20260923 22:07:44.843529 4163 raft_consensus.cc:2454] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a77b07a2e241d1821feb7b434cb90a in term 0.
I20260923 22:07:44.843675 4282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.840528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:36931 (local address 127.1.244.193:60401)
0923 22:07:44.840655 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.840666 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.840737 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.840890 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.840891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.840893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:44.840978 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.840979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.841607 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.841608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.842518 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.842520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.843155 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.843156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.843225 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.843229 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.843499 (+ 270us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.843568 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":33,"thread_start_us":62,"threads_started":1}
I20260923 22:07:44.843698 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.840551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:43171 (local address 127.1.244.193:37037)
0923 22:07:44.840681 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.840690 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.840772 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.840908 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.840908 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.840910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:44.840989 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.840991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.841611 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.841612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.842515 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.842518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.843161 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.843162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.843232 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.843236 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.843502 (+ 266us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.843550 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":45,"thread_start_us":58,"threads_started":1}
I20260923 22:07:44.843699 3956 leader_election.cc:302] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 99a77b07a2e241d1821feb7b434cb90a; no voters:
I20260923 22:07:44.843875 4013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953d2c47ce1840cbb86a5a90cc070a68" candidate_uuid: "809052d50ec74d1b9924d85c322bf3c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a77b07a2e241d1821feb7b434cb90a" is_pre_election: true
I20260923 22:07:44.843920 4163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953d2c47ce1840cbb86a5a90cc070a68" candidate_uuid: "809052d50ec74d1b9924d85c322bf3c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29739c0d28e64cad96219608f7317a6d" is_pre_election: true
I20260923 22:07:44.844013 4163 raft_consensus.cc:2454] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809052d50ec74d1b9924d85c322bf3c8 in term 0.
I20260923 22:07:44.843890 4252 raft_consensus.cc:2781] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:44.844137 4252 raft_consensus.cc:482] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:44.844161 4252 raft_consensus.cc:3037] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:44.844199 3816 leader_election.cc:302] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 809052d50ec74d1b9924d85c322bf3c8; no voters:
I20260923 22:07:44.844332 4237 raft_consensus.cc:2781] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:44.844369 4237 raft_consensus.cc:482] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:44.844388 4237 raft_consensus.cc:3037] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:44.845062 4252 raft_consensus.cc:504] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.845124 4237 raft_consensus.cc:504] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.845194 4252 leader_election.cc:288] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [CANDIDATE]: Term 1 election: Requested vote from peers 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087), 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171)
I20260923 22:07:44.845249 4237 leader_election.cc:288] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931), 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171)
I20260923 22:07:44.845263 4013 raft_consensus.cc:2379] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 809052d50ec74d1b9924d85c322bf3c8 in current term 1: Already voted for candidate 99a77b07a2e241d1821feb7b434cb90a in this term.
I20260923 22:07:44.845357 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953d2c47ce1840cbb86a5a90cc070a68" candidate_uuid: "99a77b07a2e241d1821feb7b434cb90a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8"
I20260923 22:07:44.845461 3882 raft_consensus.cc:2379] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 99a77b07a2e241d1821feb7b434cb90a in current term 1: Already voted for candidate 809052d50ec74d1b9924d85c322bf3c8 in this term.
I20260923 22:07:44.845538 4162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953d2c47ce1840cbb86a5a90cc070a68" candidate_uuid: "809052d50ec74d1b9924d85c322bf3c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29739c0d28e64cad96219608f7317a6d"
I20260923 22:07:44.845624 4162 raft_consensus.cc:3037] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:44.845615 4023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953d2c47ce1840cbb86a5a90cc070a68" candidate_uuid: "809052d50ec74d1b9924d85c322bf3c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a77b07a2e241d1821feb7b434cb90a"
I20260923 22:07:44.845691 4023 raft_consensus.cc:2379] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 809052d50ec74d1b9924d85c322bf3c8 in current term 1: Already voted for candidate 99a77b07a2e241d1821feb7b434cb90a in this term.
I20260923 22:07:44.845836 4163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953d2c47ce1840cbb86a5a90cc070a68" candidate_uuid: "99a77b07a2e241d1821feb7b434cb90a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29739c0d28e64cad96219608f7317a6d"
I20260923 22:07:44.846055 3956 leader_election.cc:302] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99a77b07a2e241d1821feb7b434cb90a; no voters: 29739c0d28e64cad96219608f7317a6d, 809052d50ec74d1b9924d85c322bf3c8
I20260923 22:07:44.846154 4252 raft_consensus.cc:2726] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260923 22:07:44.846468 4162 raft_consensus.cc:2454] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809052d50ec74d1b9924d85c322bf3c8 in term 1.
I20260923 22:07:44.846621 3816 leader_election.cc:302] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 809052d50ec74d1b9924d85c322bf3c8; no voters: 99a77b07a2e241d1821feb7b434cb90a
I20260923 22:07:44.846709 4237 raft_consensus.cc:2781] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:44.846843 4237 raft_consensus.cc:686] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 1 LEADER]: Becoming Leader. State: Replica: 809052d50ec74d1b9924d85c322bf3c8, State: Running, Role: LEADER
I20260923 22:07:44.846902 4237 consensus_queue.cc:237] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:44.847615 3725 catalog_manager.cc:5495] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "809052d50ec74d1b9924d85c322bf3c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } health_report { overall_health: UNKNOWN } } }
I20260923 22:07:44.878563 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.876537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.1.244.193:44087)
0923 22:07:44.876573 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.876575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.876605 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:07:44.876675 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.876675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.876693 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.876747 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:44.876823 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.877286 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.877974 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.878143 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.878191 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.878312 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.878401 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.878465 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.878496 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.878513 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:07:44.882825 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.880847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:39475 (local address 127.1.244.194:36931)
0923 22:07:44.880884 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.880886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.880983 (+ 97us) server_negotiation.cc:407] Connection header received
0923 22:07:44.881017 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.881018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.881026 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.881072 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:44.881155 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.881582 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.882257 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.882422 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.882469 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.882623 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.882646 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.882686 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.882756 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.882777 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:07:44.882843 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.880776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:36931 (local address 127.1.244.193:39475)
0923 22:07:44.880907 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.880925 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.880980 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.881085 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.881085 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.881086 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:07:44.881117 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.881118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.881633 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.881634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.882191 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.882192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.882500 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.882501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.882544 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.882556 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.882715 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.882717 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.882753 (+ 36us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.882784 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260923 22:07:44.884938 4023 raft_consensus.cc:1261] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Refusing update from remote peer 809052d50ec74d1b9924d85c322bf3c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:44.884974 4162 raft_consensus.cc:1261] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Refusing update from remote peer 809052d50ec74d1b9924d85c322bf3c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:44.885231 4237 consensus_queue.cc:1035] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:44.885401 4283 consensus_queue.cc:1035] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:44.892235 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.890095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:52541 (local address 127.1.244.193:44087)
0923 22:07:44.890154 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:44.890156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:44.890164 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:07:44.890184 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:44.890185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:44.890193 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:44.890236 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:44.890329 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.890810 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.891511 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.891807 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.891877 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.892023 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.892054 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.892114 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:44.892178 (+ 64us) server_negotiation.cc:299] Negotiation successful
0923 22:07:44.892193 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:07:44.892302 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:44.889957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44087 (local address 127.1.244.194:52541)
0923 22:07:44.889993 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:44.890003 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:44.890038 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:44.890248 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:44.890249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:44.890250 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:07:44.890285 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.890286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.890893 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.890894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.891426 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:44.891427 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:44.891888 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:44.891889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:44.891942 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:44.891948 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:07:44.892118 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:07:44.892120 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:07:44.892171 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:07:44.892220 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
W20260923 22:07:44.892601 4258 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 22:07:46.411851 3726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:46.411958 3726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:46.412731 3726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553: Sending DeleteTablet for 3 replicas of tablet 953d2c47ce1840cbb86a5a90cc070a68
I20260923 22:07:46.412989 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet 953d2c47ce1840cbb86a5a90cc070a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:46 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46704
I20260923 22:07:46.413041 3848 tablet_service.cc:1514] Processing DeleteTablet for tablet 953d2c47ce1840cbb86a5a90cc070a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:46 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:50508
I20260923 22:07:46.413224 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet 953d2c47ce1840cbb86a5a90cc070a68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:46 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33318
I20260923 22:07:46.413391 4312 tablet_replica.cc:330] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: stopping tablet replica
I20260923 22:07:46.413450 4312 raft_consensus.cc:2229] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:46.413482 4312 raft_consensus.cc:2258] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:46.413561 4313 tablet_replica.cc:330] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: stopping tablet replica
I20260923 22:07:46.413614 4313 raft_consensus.cc:2229] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:07:46.413683 4313 raft_consensus.cc:2258] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:46.413771 4314 tablet_replica.cc:330] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: stopping tablet replica
I20260923 22:07:46.413820 4314 raft_consensus.cc:2229] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:46.413862 4314 raft_consensus.cc:2258] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:46.414152 4312 ts_tablet_manager.cc:1905] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:46.415759 4314 ts_tablet_manager.cc:1905] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:46.415913 4312 ts_tablet_manager.cc:1918] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:07:46.415997 4312 log.cc:1196] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/953d2c47ce1840cbb86a5a90cc070a68
I20260923 22:07:46.416213 4312 ts_tablet_manager.cc:1939] T 953d2c47ce1840cbb86a5a90cc070a68 P 99a77b07a2e241d1821feb7b434cb90a: Deleting consensus metadata
I20260923 22:07:46.416499 3711 catalog_manager.cc:4841] TS 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931): tablet 953d2c47ce1840cbb86a5a90cc070a68 (table test-table [id=af478942dcca4ca284907361331a5b63]) successfully deleted
I20260923 22:07:46.417752 4314 ts_tablet_manager.cc:1918] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:07:46.417812 4314 log.cc:1196] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/953d2c47ce1840cbb86a5a90cc070a68
I20260923 22:07:46.418005 4314 ts_tablet_manager.cc:1939] T 953d2c47ce1840cbb86a5a90cc070a68 P 29739c0d28e64cad96219608f7317a6d: Deleting consensus metadata
I20260923 22:07:46.418339 3712 catalog_manager.cc:4841] TS 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171): tablet 953d2c47ce1840cbb86a5a90cc070a68 (table test-table [id=af478942dcca4ca284907361331a5b63]) successfully deleted
I20260923 22:07:46.418358 4313 ts_tablet_manager.cc:1905] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:46.419489 4313 ts_tablet_manager.cc:1918] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:07:46.419582 4313 log.cc:1196] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/953d2c47ce1840cbb86a5a90cc070a68
I20260923 22:07:46.419813 4313 ts_tablet_manager.cc:1939] T 953d2c47ce1840cbb86a5a90cc070a68 P 809052d50ec74d1b9924d85c322bf3c8: Deleting consensus metadata
I20260923 22:07:46.420147 3712 catalog_manager.cc:4841] TS 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087): tablet 953d2c47ce1840cbb86a5a90cc070a68 (table test-table [id=af478942dcca4ca284907361331a5b63]) successfully deleted
Sep 23 22:07:46 dist-test-slave-b5jz krb5kdc[3681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201266, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:07:46 dist-test-slave-b5jz krb5kdc[3681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201266, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:46.435642 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:46.428688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.1.244.254:39595)
0923 22:07:46.428802 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:46.428804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:46.428813 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:07:46.428884 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:46.428885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:46.428897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:46.428979 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:46.429511 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:46.430002 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:46.430727 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:46.430929 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:46.431005 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:46.433171 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:46.433173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:46.433173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:46.433180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:46.434481 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:46.434825 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:46.434826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:46.434826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:46.434885 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:46.435100 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:46.435100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:46.435101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:46.435207 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:46.435312 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:46.435416 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 22:07:46.435498 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":59,"threads_started":1}
I20260923 22:07:46.437036 3726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:07:46.437187 3726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:46.438496 3848 tablet_service.cc:1467] Processing CreateTablet for tablet baca612113f5404eb40c47203bbe8fde (DEFAULT_TABLE table=test-table [id=0f46597cf92641eaa5e05cc91a0df82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:46.438524 4003 tablet_service.cc:1467] Processing CreateTablet for tablet baca612113f5404eb40c47203bbe8fde (DEFAULT_TABLE table=test-table [id=0f46597cf92641eaa5e05cc91a0df82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:46.438625 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baca612113f5404eb40c47203bbe8fde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:46.438628 3848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baca612113f5404eb40c47203bbe8fde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:46.439985 4324 tablet_bootstrap.cc:492] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: Bootstrap starting.
I20260923 22:07:46.439980 4143 tablet_service.cc:1467] Processing CreateTablet for tablet baca612113f5404eb40c47203bbe8fde (DEFAULT_TABLE table=test-table [id=0f46597cf92641eaa5e05cc91a0df82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:46.440050 4323 tablet_bootstrap.cc:492] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: Bootstrap starting.
I20260923 22:07:46.440099 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baca612113f5404eb40c47203bbe8fde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:46.440380 4323 tablet_bootstrap.cc:654] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:46.440382 4324 tablet_bootstrap.cc:654] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:46.441234 4324 tablet_bootstrap.cc:492] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: No bootstrap required, opened a new log
I20260923 22:07:46.441267 4324 ts_tablet_manager.cc:1397] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:46.441344 4324 raft_consensus.cc:348] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.441390 4324 raft_consensus.cc:374] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:46.441401 4324 raft_consensus.cc:729] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809052d50ec74d1b9924d85c322bf3c8, State: Initialized, Role: FOLLOWER
I20260923 22:07:46.441422 4324 consensus_queue.cc:260] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.441533 4324 ts_tablet_manager.cc:1428] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:46.441743 4327 tablet_bootstrap.cc:492] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: Bootstrap starting.
I20260923 22:07:46.442090 4327 tablet_bootstrap.cc:654] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:46.442638 4327 tablet_bootstrap.cc:492] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: No bootstrap required, opened a new log
I20260923 22:07:46.442646 4323 tablet_bootstrap.cc:492] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: No bootstrap required, opened a new log
I20260923 22:07:46.442701 4323 ts_tablet_manager.cc:1397] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:07:46.442706 4327 ts_tablet_manager.cc:1397] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:46.442830 4323 raft_consensus.cc:348] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.442842 4327 raft_consensus.cc:348] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.442891 4323 raft_consensus.cc:374] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:46.442911 4323 raft_consensus.cc:729] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a77b07a2e241d1821feb7b434cb90a, State: Initialized, Role: FOLLOWER
I20260923 22:07:46.442914 4327 raft_consensus.cc:374] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:46.442937 4327 raft_consensus.cc:729] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29739c0d28e64cad96219608f7317a6d, State: Initialized, Role: FOLLOWER
I20260923 22:07:46.442976 4323 consensus_queue.cc:260] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.442996 4327 consensus_queue.cc:260] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.443079 4323 ts_tablet_manager.cc:1428] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:46.443275 4327 ts_tablet_manager.cc:1428] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:46.636626 4329 raft_consensus.cc:482] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:46.636750 4329 raft_consensus.cc:504] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.636999 4329 leader_election.cc:288] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087), 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931)
I20260923 22:07:46.637183 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baca612113f5404eb40c47203bbe8fde" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8" is_pre_election: true
I20260923 22:07:46.637217 4023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baca612113f5404eb40c47203bbe8fde" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a77b07a2e241d1821feb7b434cb90a" is_pre_election: true
I20260923 22:07:46.637292 3882 raft_consensus.cc:2454] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 0.
I20260923 22:07:46.637316 4023 raft_consensus.cc:2454] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 0.
I20260923 22:07:46.637472 4096 leader_election.cc:302] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 809052d50ec74d1b9924d85c322bf3c8; no voters:
I20260923 22:07:46.637584 4329 raft_consensus.cc:2781] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:46.637624 4329 raft_consensus.cc:482] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:46.637642 4329 raft_consensus.cc:3037] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:46.638368 4329 raft_consensus.cc:504] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.638520 4329 leader_election.cc:288] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 election: Requested vote from peers 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087), 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931)
I20260923 22:07:46.638679 4023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baca612113f5404eb40c47203bbe8fde" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a77b07a2e241d1821feb7b434cb90a"
I20260923 22:07:46.638697 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baca612113f5404eb40c47203bbe8fde" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8"
I20260923 22:07:46.638759 3882 raft_consensus.cc:3037] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:46.638767 4023 raft_consensus.cc:3037] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:46.639429 3882 raft_consensus.cc:2454] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 1.
I20260923 22:07:46.639431 4023 raft_consensus.cc:2454] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 1.
I20260923 22:07:46.639611 4095 leader_election.cc:302] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 99a77b07a2e241d1821feb7b434cb90a; no voters:
I20260923 22:07:46.639772 4329 raft_consensus.cc:2781] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:46.639906 4329 raft_consensus.cc:686] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 1 LEADER]: Becoming Leader. State: Replica: 29739c0d28e64cad96219608f7317a6d, State: Running, Role: LEADER
I20260923 22:07:46.639973 4329 consensus_queue.cc:237] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.640578 3725 catalog_manager.cc:5495] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 29739c0d28e64cad96219608f7317a6d (127.1.244.195). New cstate: current_term: 1 leader_uuid: "29739c0d28e64cad96219608f7317a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } health_report { overall_health: HEALTHY } } }
I20260923 22:07:46.669579 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:46.667394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33334 (local address 127.1.244.195:43171)
0923 22:07:46.667590 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:46.667592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:46.667599 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:07:46.667617 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:46.667618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:46.667627 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:46.667683 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:46.667801 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:46.668267 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:46.668969 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:46.669153 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:46.669205 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:46.669305 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:46.669351 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:46.669385 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:46.669435 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 22:07:46.669451 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":109,"threads_started":1}
I20260923 22:07:46.671777 4023 raft_consensus.cc:1261] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Refusing update from remote peer 29739c0d28e64cad96219608f7317a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:46.671778 3882 raft_consensus.cc:1261] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Refusing update from remote peer 29739c0d28e64cad96219608f7317a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:46.671998 4329 consensus_queue.cc:1035] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:46.672092 4330 consensus_queue.cc:1035] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:46.681660 3726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:46.681742 3726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:46.682291 3726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553: Sending DeleteTablet for 3 replicas of tablet baca612113f5404eb40c47203bbe8fde
I20260923 22:07:46.682538 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet baca612113f5404eb40c47203bbe8fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:46 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46704
I20260923 22:07:46.682569 3848 tablet_service.cc:1514] Processing DeleteTablet for tablet baca612113f5404eb40c47203bbe8fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:46 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:50508
I20260923 22:07:46.682600 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet baca612113f5404eb40c47203bbe8fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:46 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33318
I20260923 22:07:46.682628 4312 tablet_replica.cc:330] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: stopping tablet replica
I20260923 22:07:46.682674 4312 raft_consensus.cc:2229] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:46.682675 4314 tablet_replica.cc:330] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: stopping tablet replica
I20260923 22:07:46.682714 4312 raft_consensus.cc:2258] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:46.682724 4314 raft_consensus.cc:2229] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:07:46.682772 4314 raft_consensus.cc:2258] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:46.682796 4313 tablet_replica.cc:330] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: stopping tablet replica
I20260923 22:07:46.682857 4313 raft_consensus.cc:2229] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:46.682886 4313 raft_consensus.cc:2258] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:46.683208 4314 ts_tablet_manager.cc:1905] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:46.683336 3726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:07:46.683406 4312 ts_tablet_manager.cc:1905] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:46.683415 4313 ts_tablet_manager.cc:1905] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:07:46.683475 3726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:46.684970 4313 ts_tablet_manager.cc:1918] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:07:46.685025 4313 log.cc:1196] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/baca612113f5404eb40c47203bbe8fde
I20260923 22:07:46.685029 4312 ts_tablet_manager.cc:1918] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:07:46.685076 4312 log.cc:1196] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/baca612113f5404eb40c47203bbe8fde
I20260923 22:07:46.685209 4314 ts_tablet_manager.cc:1918] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:07:46.685259 4314 log.cc:1196] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/baca612113f5404eb40c47203bbe8fde
I20260923 22:07:46.685263 4313 ts_tablet_manager.cc:1939] T baca612113f5404eb40c47203bbe8fde P 809052d50ec74d1b9924d85c322bf3c8: Deleting consensus metadata
I20260923 22:07:46.685302 4312 ts_tablet_manager.cc:1939] T baca612113f5404eb40c47203bbe8fde P 99a77b07a2e241d1821feb7b434cb90a: Deleting consensus metadata
I20260923 22:07:46.685461 4314 ts_tablet_manager.cc:1939] T baca612113f5404eb40c47203bbe8fde P 29739c0d28e64cad96219608f7317a6d: Deleting consensus metadata
I20260923 22:07:46.685755 3712 catalog_manager.cc:4841] TS 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171): tablet baca612113f5404eb40c47203bbe8fde (table test-table [id=0f46597cf92641eaa5e05cc91a0df82d]) successfully deleted
I20260923 22:07:46.686951 3848 tablet_service.cc:1467] Processing CreateTablet for tablet 62bc093223ed4596b6dadf155a8c4fdc (DEFAULT_TABLE table=test-table [id=79684790d4934c91af2847932876ca56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:46.687028 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 62bc093223ed4596b6dadf155a8c4fdc (DEFAULT_TABLE table=test-table [id=79684790d4934c91af2847932876ca56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:46.687069 3848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62bc093223ed4596b6dadf155a8c4fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:46.687125 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62bc093223ed4596b6dadf155a8c4fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:46.688258 4327 tablet_bootstrap.cc:492] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: Bootstrap starting.
I20260923 22:07:46.688269 4324 tablet_bootstrap.cc:492] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8: Bootstrap starting.
I20260923 22:07:46.688454 3712 catalog_manager.cc:4841] TS 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087): tablet baca612113f5404eb40c47203bbe8fde (table test-table [id=0f46597cf92641eaa5e05cc91a0df82d]) successfully deleted
I20260923 22:07:46.688591 4324 tablet_bootstrap.cc:654] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:46.688647 4327 tablet_bootstrap.cc:654] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:46.688957 3711 catalog_manager.cc:4841] TS 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931): tablet baca612113f5404eb40c47203bbe8fde (table test-table [id=0f46597cf92641eaa5e05cc91a0df82d]) successfully deleted
I20260923 22:07:46.689240 4324 tablet_bootstrap.cc:492] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8: No bootstrap required, opened a new log
I20260923 22:07:46.689298 4324 ts_tablet_manager.cc:1397] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:46.689400 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 62bc093223ed4596b6dadf155a8c4fdc (DEFAULT_TABLE table=test-table [id=79684790d4934c91af2847932876ca56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:46.689489 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62bc093223ed4596b6dadf155a8c4fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:46.689438 4324 raft_consensus.cc:348] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.689514 4324 raft_consensus.cc:374] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:46.689543 4324 raft_consensus.cc:729] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809052d50ec74d1b9924d85c322bf3c8, State: Initialized, Role: FOLLOWER
I20260923 22:07:46.689597 4324 consensus_queue.cc:260] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.689707 4324 ts_tablet_manager.cc:1428] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:46.690478 4327 tablet_bootstrap.cc:492] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: No bootstrap required, opened a new log
I20260923 22:07:46.690531 4327 ts_tablet_manager.cc:1397] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:46.690624 4327 raft_consensus.cc:348] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.690701 4327 raft_consensus.cc:374] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:46.690721 4327 raft_consensus.cc:729] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29739c0d28e64cad96219608f7317a6d, State: Initialized, Role: FOLLOWER
I20260923 22:07:46.690778 4327 consensus_queue.cc:260] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.690872 4327 ts_tablet_manager.cc:1428] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:46.690943 4323 tablet_bootstrap.cc:492] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: Bootstrap starting.
I20260923 22:07:46.691282 4323 tablet_bootstrap.cc:654] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:46.692451 4323 tablet_bootstrap.cc:492] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: No bootstrap required, opened a new log
I20260923 22:07:46.692504 4323 ts_tablet_manager.cc:1397] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:46.692618 4323 raft_consensus.cc:348] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.692706 4323 raft_consensus.cc:374] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:46.692734 4323 raft_consensus.cc:729] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a77b07a2e241d1821feb7b434cb90a, State: Initialized, Role: FOLLOWER
I20260923 22:07:46.692775 4323 consensus_queue.cc:260] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.692865 4323 ts_tablet_manager.cc:1428] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:46.715649 4337 raft_consensus.cc:482] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:46.715734 4337 raft_consensus.cc:504] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.715891 4337 leader_election.cc:288] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087), 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931)
I20260923 22:07:46.716069 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62bc093223ed4596b6dadf155a8c4fdc" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8" is_pre_election: true
I20260923 22:07:46.716068 4023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62bc093223ed4596b6dadf155a8c4fdc" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a77b07a2e241d1821feb7b434cb90a" is_pre_election: true
I20260923 22:07:46.716176 3882 raft_consensus.cc:2454] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 0.
I20260923 22:07:46.716193 4023 raft_consensus.cc:2454] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 0.
I20260923 22:07:46.716362 4095 leader_election.cc:302] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 99a77b07a2e241d1821feb7b434cb90a; no voters:
I20260923 22:07:46.716470 4337 raft_consensus.cc:2781] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:46.716519 4337 raft_consensus.cc:482] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:46.716544 4337 raft_consensus.cc:3037] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:46.717271 4337 raft_consensus.cc:504] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.717413 4337 leader_election.cc:288] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 election: Requested vote from peers 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087), 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931)
I20260923 22:07:46.717535 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62bc093223ed4596b6dadf155a8c4fdc" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809052d50ec74d1b9924d85c322bf3c8"
I20260923 22:07:46.717577 4023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62bc093223ed4596b6dadf155a8c4fdc" candidate_uuid: "29739c0d28e64cad96219608f7317a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a77b07a2e241d1821feb7b434cb90a"
I20260923 22:07:46.717607 3882 raft_consensus.cc:3037] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:46.717629 4023 raft_consensus.cc:3037] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:46.718326 4023 raft_consensus.cc:2454] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 1.
I20260923 22:07:46.718326 3882 raft_consensus.cc:2454] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29739c0d28e64cad96219608f7317a6d in term 1.
I20260923 22:07:46.718538 4095 leader_election.cc:302] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29739c0d28e64cad96219608f7317a6d, 99a77b07a2e241d1821feb7b434cb90a; no voters:
I20260923 22:07:46.718647 4337 raft_consensus.cc:2781] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:46.718744 4337 raft_consensus.cc:686] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 1 LEADER]: Becoming Leader. State: Replica: 29739c0d28e64cad96219608f7317a6d, State: Running, Role: LEADER
I20260923 22:07:46.718802 4337 consensus_queue.cc:237] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } }
I20260923 22:07:46.719357 3726 catalog_manager.cc:5495] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 29739c0d28e64cad96219608f7317a6d (127.1.244.195). New cstate: current_term: 1 leader_uuid: "29739c0d28e64cad96219608f7317a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29739c0d28e64cad96219608f7317a6d" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43171 } health_report { overall_health: HEALTHY } } }
I20260923 22:07:46.728538 4023 raft_consensus.cc:1261] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Refusing update from remote peer 29739c0d28e64cad96219608f7317a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:46.728579 3882 raft_consensus.cc:1261] T 62bc093223ed4596b6dadf155a8c4fdc P 809052d50ec74d1b9924d85c322bf3c8 [term 1 FOLLOWER]: Refusing update from remote peer 29739c0d28e64cad96219608f7317a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:46.728753 4329 consensus_queue.cc:1035] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a77b07a2e241d1821feb7b434cb90a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:46.728843 4337 consensus_queue.cc:1035] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "809052d50ec74d1b9924d85c322bf3c8" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:07:46.732659 4296 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 22:07:48.248876 3725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:48.248989 3725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:48.249863 3725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a74380df5c364aa4b846eefad8bc2553: Sending DeleteTablet for 3 replicas of tablet 62bc093223ed4596b6dadf155a8c4fdc
I20260923 22:07:48.250126 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet 62bc093223ed4596b6dadf155a8c4fdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:48 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33318
I20260923 22:07:48.250134 3848 tablet_service.cc:1514] Processing DeleteTablet for tablet 62bc093223ed4596b6dadf155a8c4fdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:48 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:50508
I20260923 22:07:48.250134 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet 62bc093223ed4596b6dadf155a8c4fdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:48 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46704
I20260923 22:07:48.250326 4366 tablet_replica.cc:330] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: stopping tablet replica
I20260923 22:07:48.250427 4366 raft_consensus.cc:2229] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:07:48.250442 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 3798
I20260923 22:07:48.250490 4366 raft_consensus.cc:2258] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:48.251228 4366 ts_tablet_manager.cc:1905] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:48.252579 4367 tablet_replica.cc:330] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: stopping tablet replica
I20260923 22:07:48.252679 4367 raft_consensus.cc:2229] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:48.252724 4367 raft_consensus.cc:2258] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:48.252882 4366 ts_tablet_manager.cc:1918] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:07:48.252939 4366 log.cc:1196] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/62bc093223ed4596b6dadf155a8c4fdc
I20260923 22:07:48.252966 4367 ts_tablet_manager.cc:1905] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:48.253245 4366 ts_tablet_manager.cc:1939] T 62bc093223ed4596b6dadf155a8c4fdc P 29739c0d28e64cad96219608f7317a6d: Deleting consensus metadata
I20260923 22:07:48.254374 4367 ts_tablet_manager.cc:1918] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:07:48.254451 4367 log.cc:1196] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/62bc093223ed4596b6dadf155a8c4fdc
I20260923 22:07:48.254658 4367 ts_tablet_manager.cc:1939] T 62bc093223ed4596b6dadf155a8c4fdc P 99a77b07a2e241d1821feb7b434cb90a: Deleting consensus metadata
I20260923 22:07:48.254980 3711 catalog_manager.cc:4841] TS 99a77b07a2e241d1821feb7b434cb90a (127.1.244.194:36931): tablet 62bc093223ed4596b6dadf155a8c4fdc (table test-table [id=79684790d4934c91af2847932876ca56]) successfully deleted
I20260923 22:07:48.255656 3712 catalog_manager.cc:4841] TS 29739c0d28e64cad96219608f7317a6d (127.1.244.195:43171): tablet 62bc093223ed4596b6dadf155a8c4fdc (table test-table [id=79684790d4934c91af2847932876ca56]) successfully deleted
W20260923 22:07:48.257112 3712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:44087 (error 108)
I20260923 22:07:48.257247 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 3938
I20260923 22:07:48.258515 4370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:48.257493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44087 (local address 127.0.0.1:50520)
0923 22:07:48.258443 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:48.258486 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:44087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":57,"threads_started":1}
W20260923 22:07:48.258662 3712 catalog_manager.cc:4584] TS 809052d50ec74d1b9924d85c322bf3c8 (127.1.244.193:44087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62bc093223ed4596b6dadf155a8c4fdc: Network error: Client connection negotiation failed: client connection to 127.1.244.193:44087: connect: Connection refused (error 111)
I20260923 22:07:48.263456 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 4078
I20260923 22:07:48.268772 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 3694
2026-09-23T22:07:48Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5873 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:48 dist-test-slave-b5jz krb5kdc[4373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:48 dist-test-slave-b5jz krb5kdc[4373](info): set up 2 sockets
Sep 23 22:07:48 dist-test-slave-b5jz krb5kdc[4373](info): commencing operation
krb5kdc: starting...
W20260923 22:07:49.335467 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:07:49 dist-test-slave-b5jz krb5kdc[4373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:07:49.454337 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42887
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:42887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:49.534507 4386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:49.534754 4386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:49.534796 4386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:49.535967 4386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:49.536041 4386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:49.536065 4386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:49.536096 4386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:49.536127 4386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:49.537622 4386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:42887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:49.537902 4386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:49.538149 4386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:49.543133 4394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:49.543139 4391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:49.543155 4386 server_base.cc:1034] running on GCE node
W20260923 22:07:49.543139 4392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:49.543716 4386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:49.544077 4386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:49.545256 4386 hybrid_clock.cc:648] HybridClock initialized: now 1790201269545234 us; error 39 us; skew 500 ppm
Sep 23 22:07:49 dist-test-slave-b5jz krb5kdc[4373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201269, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:49.553795 4386 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:07:49.554337 4386 webserver.cc:460] Webserver started at http://127.1.244.254:40199/ using document root <none> and password file <none>
I20260923 22:07:49.554615 4386 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:49.554692 4386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:49.554847 4386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:49.556031 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "c4b2ee1912c8414fb6d0776982e61604"
format_stamp: "Formatted at 2026-09-23 22:07:49 on dist-test-slave-b5jz"
server_key: "6e633d3041d3aa339cad12d450541933"
server_key_iv: "1b33dc83f18bfa18905ec566a4da8b9d"
server_key_version: "encryptionkey@0"
I20260923 22:07:49.556393 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "c4b2ee1912c8414fb6d0776982e61604"
format_stamp: "Formatted at 2026-09-23 22:07:49 on dist-test-slave-b5jz"
server_key: "6e633d3041d3aa339cad12d450541933"
server_key_iv: "1b33dc83f18bfa18905ec566a4da8b9d"
server_key_version: "encryptionkey@0"
I20260923 22:07:49.557742 4386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:49.558514 4401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:49.558753 4386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:49.558861 4386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "c4b2ee1912c8414fb6d0776982e61604"
format_stamp: "Formatted at 2026-09-23 22:07:49 on dist-test-slave-b5jz"
server_key: "6e633d3041d3aa339cad12d450541933"
server_key_iv: "1b33dc83f18bfa18905ec566a4da8b9d"
server_key_version: "encryptionkey@0"
I20260923 22:07:49.558936 4386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:49.595786 4386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:49.597798 4386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:49.597970 4386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:49.602116 4386 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:42887
I20260923 22:07:49.602175 4453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:42887 every 8 connection(s)
I20260923 22:07:49.602562 4386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:49.603457 4454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:49.605791 4454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604: Bootstrap starting.
I20260923 22:07:49.606321 4454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:49.606562 4454 log.cc:824] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:49.607241 4454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604: No bootstrap required, opened a new log
I20260923 22:07:49.608857 4454 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b2ee1912c8414fb6d0776982e61604" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42887 } }
I20260923 22:07:49.609000 4454 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:49.609023 4454 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b2ee1912c8414fb6d0776982e61604, State: Initialized, Role: FOLLOWER
I20260923 22:07:49.609172 4454 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b2ee1912c8414fb6d0776982e61604" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42887 } }
I20260923 22:07:49.609256 4454 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:49.609280 4454 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:49.609309 4454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:49.609531 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 4386
I20260923 22:07:49.609632 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:07:49.609819 2003 external_mini_cluster.cc:1428] Setting key 4449171a6bf98019b68738fe7a7e3319
I20260923 22:07:49.609863 4454 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b2ee1912c8414fb6d0776982e61604" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42887 } }
I20260923 22:07:49.610013 4454 leader_election.cc:302] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4b2ee1912c8414fb6d0776982e61604; no voters:
I20260923 22:07:49.610211 4454 leader_election.cc:288] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:49.610464 4457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:49.610519 4454 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:49.610682 4457 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [term 1 LEADER]: Becoming Leader. State: Replica: c4b2ee1912c8414fb6d0776982e61604, State: Running, Role: LEADER
I20260923 22:07:49.610816 4457 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b2ee1912c8414fb6d0776982e61604" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42887 } }
I20260923 22:07:49.611900 4459 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4b2ee1912c8414fb6d0776982e61604. Latest consensus state: current_term: 1 leader_uuid: "c4b2ee1912c8414fb6d0776982e61604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b2ee1912c8414fb6d0776982e61604" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42887 } } }
I20260923 22:07:49.611999 4459 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:49.612205 4474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:49.612679 4474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:49.613462 4461 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4b2ee1912c8414fb6d0776982e61604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b2ee1912c8414fb6d0776982e61604" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42887 } } }
I20260923 22:07:49.613557 4461 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:49.614271 4474 catalog_manager.cc:1347] Generated new cluster ID: 36b3472a6fbf4d11878eb61883c6feac
I20260923 22:07:49.614336 4474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:07:49 dist-test-slave-b5jz krb5kdc[4373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:49.622185 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:49.610295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55854 (local address 127.1.244.254:42887)
0923 22:07:49.610511 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:49.610520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:49.610632 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:07:49.610764 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:49.610771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:49.610829 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:49.611011 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:49.611795 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.613184 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.614245 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.614844 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.614961 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:49.617596 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:49.617600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:49.617600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:49.617613 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:49.620383 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:49.620779 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:49.620782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:49.620783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:49.620896 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:49.621104 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:49.621105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:49.621105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:49.621441 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:49.621617 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:49.621689 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:07:49.621947 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":78,"threads_started":1}
I20260923 22:07:49.624169 4474 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:49.624671 4474 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:49.631798 4474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4b2ee1912c8414fb6d0776982e61604: Generated new TSK 0
I20260923 22:07:49.632022 4474 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:07:49.662832 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42887
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:49.769191 4480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:49.769418 4480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:49.769443 4480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:49.770620 4480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:49.770660 4480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:49.770686 4480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:07:49.772068 4480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:49.772253 4480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:49.772428 4480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:49.777537 4488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:49.777539 4485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:49.777645 4486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:49.777927 4480 server_base.cc:1034] running on GCE node
I20260923 22:07:49.778159 4480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:49.778401 4480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:49.779593 4480 hybrid_clock.cc:648] HybridClock initialized: now 1790201269779574 us; error 27 us; skew 500 ppm
Sep 23 22:07:49 dist-test-slave-b5jz krb5kdc[4373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201269, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:49.782198 4480 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:07:49.782611 4480 webserver.cc:460] Webserver started at http://127.1.244.193:42877/ using document root <none> and password file <none>
I20260923 22:07:49.782791 4480 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:49.782832 4480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:49.782920 4480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:49.784003 4480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "a16b6843ae3f476ea436d7d85b9dbbcc"
format_stamp: "Formatted at 2026-09-23 22:07:49 on dist-test-slave-b5jz"
server_key: "49389a706971ac5eda63b97f7d35cf08"
server_key_iv: "4e3fe7fdce01a74683fec3a2fe8017af"
server_key_version: "encryptionkey@0"
I20260923 22:07:49.784302 4480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "a16b6843ae3f476ea436d7d85b9dbbcc"
format_stamp: "Formatted at 2026-09-23 22:07:49 on dist-test-slave-b5jz"
server_key: "49389a706971ac5eda63b97f7d35cf08"
server_key_iv: "4e3fe7fdce01a74683fec3a2fe8017af"
server_key_version: "encryptionkey@0"
I20260923 22:07:49.785545 4480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:49.786255 4495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:49.786427 4480 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:49.786512 4480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "a16b6843ae3f476ea436d7d85b9dbbcc"
format_stamp: "Formatted at 2026-09-23 22:07:49 on dist-test-slave-b5jz"
server_key: "49389a706971ac5eda63b97f7d35cf08"
server_key_iv: "4e3fe7fdce01a74683fec3a2fe8017af"
server_key_version: "encryptionkey@0"
I20260923 22:07:49.786569 4480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:49.798120 4480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:49.800539 4480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:49.800668 4480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:49.800860 4480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:49.801203 4480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:49.801240 4480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:49.801265 4480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:49.801278 4480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:49.807155 4480 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:34641
I20260923 22:07:49.807230 4608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:34641 every 8 connection(s)
I20260923 22:07:49.807634 4480 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:07:49.808604 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 4480
I20260923 22:07:49.808718 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:49.808905 2003 external_mini_cluster.cc:1428] Setting key 6312b05a435b8674f0499355571fe522
Sep 23 22:07:49 dist-test-slave-b5jz krb5kdc[4373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201269, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260923 22:07:49.818794 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:49.808500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:56969 (local address 127.1.244.254:42887)
0923 22:07:49.808555 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:49.808557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:49.808768 (+ 211us) server_negotiation.cc:407] Connection header received
0923 22:07:49.809444 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:49.809445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:49.809455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:49.809541 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:49.810666 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.811318 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.812722 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.813085 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.813168 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:49.815539 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:49.815540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:49.815541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:49.815579 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:49.817316 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:49.817792 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:49.817793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:49.817793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:49.817861 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:49.818095 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:49.818096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:49.818096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:49.818205 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:49.818290 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:49.818645 (+ 355us) server_negotiation.cc:299] Negotiation successful
0923 22:07:49.818725 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:07:49.818935 4611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:49.808475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42887 (local address 127.1.244.193:56969)
0923 22:07:49.808643 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:49.808687 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:49.809344 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:49.809637 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:49.809642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:49.810348 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:49.810559 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:49.810564 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.811420 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.811421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:49.812638 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:49.812640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.813221 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.813222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:49.813366 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:49.814004 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:49.814014 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:49.815437 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:49.817423 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:49.817431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:49.817434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:49.817718 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:49.817929 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:49.817930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:49.817930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:49.818038 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:49.818326 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:49.818335 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:49.818483 (+ 148us) client_negotiation.cc:769] Sending connection context
0923 22:07:49.818585 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 22:07:49.818731 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260923 22:07:49.819525 4609 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42887
I20260923 22:07:49.819706 4609 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
I20260923 22:07:49.819947 4609 heartbeater.cc:511] Master 127.1.244.254:42887 requested a full tablet report, sending...
I20260923 22:07:49.820560 4418 ts_manager.cc:194] Registered new tserver with Master: a16b6843ae3f476ea436d7d85b9dbbcc (127.1.244.193:34641)
I20260923 22:07:49.821192 4418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:56969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:07:49.834856 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42887
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:49.921482 4614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:49.921743 4614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:49.921782 4614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:49.922966 4614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:49.923058 4614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:49.923103 4614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:07:49.924594 4614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:49.924876 4614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:49.925122 4614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:49.930308 4619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:49.930346 4620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:49.930346 4622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:49.930402 4614 server_base.cc:1034] running on GCE node
I20260923 22:07:49.930795 4614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:49.931051 4614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:49.932231 4614 hybrid_clock.cc:648] HybridClock initialized: now 1790201269932230 us; error 51 us; skew 500 ppm
Sep 23 22:07:49 dist-test-slave-b5jz krb5kdc[4373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201269, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:49.934728 4614 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:07:49.935230 4614 webserver.cc:460] Webserver started at http://127.1.244.194:43615/ using document root <none> and password file <none>
I20260923 22:07:49.935467 4614 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:49.935535 4614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:49.935726 4614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:49.936828 4614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "0e4b6279f07f4adbab347372def770e0"
format_stamp: "Formatted at 2026-09-23 22:07:49 on dist-test-slave-b5jz"
server_key: "706acc0d85ad63c744de28013833a1b3"
server_key_iv: "dede73b9b17941d79705e4a7faf620ed"
server_key_version: "encryptionkey@0"
I20260923 22:07:49.937228 4614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "0e4b6279f07f4adbab347372def770e0"
format_stamp: "Formatted at 2026-09-23 22:07:49 on dist-test-slave-b5jz"
server_key: "706acc0d85ad63c744de28013833a1b3"
server_key_iv: "dede73b9b17941d79705e4a7faf620ed"
server_key_version: "encryptionkey@0"
I20260923 22:07:49.938594 4614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:07:49.939430 4629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:49.939700 4614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:49.939795 4614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "0e4b6279f07f4adbab347372def770e0"
format_stamp: "Formatted at 2026-09-23 22:07:49 on dist-test-slave-b5jz"
server_key: "706acc0d85ad63c744de28013833a1b3"
server_key_iv: "dede73b9b17941d79705e4a7faf620ed"
server_key_version: "encryptionkey@0"
I20260923 22:07:49.939862 4614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:49.957208 4614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:49.959188 4614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:49.959303 4614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:49.959493 4614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:49.959862 4614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:49.959897 4614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:49.959921 4614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:49.959933 4614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:49.965786 4614 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:40193
I20260923 22:07:49.965832 4742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:40193 every 8 connection(s)
I20260923 22:07:49.966166 4614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:07:49.970348 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 4614
I20260923 22:07:49.970439 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:49.970656 2003 external_mini_cluster.cc:1428] Setting key 5a40e627af8749ed6ef4022b12198b99
Sep 23 22:07:49 dist-test-slave-b5jz krb5kdc[4373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201269, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:49.976795 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:49.966930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:33461 (local address 127.1.244.254:42887)
0923 22:07:49.967021 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:49.967024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:49.967277 (+ 253us) server_negotiation.cc:407] Connection header received
0923 22:07:49.967844 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:49.967845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:49.967857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:49.967916 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:49.968865 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.969366 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.970542 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.970842 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.970953 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:49.973297 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:49.973299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:49.973299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:49.973307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:49.975090 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:49.975705 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:49.975707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:49.975707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:49.975789 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:49.976103 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:49.976104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:49.976104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:49.976235 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:49.976307 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:49.976607 (+ 300us) server_negotiation.cc:299] Negotiation successful
0923 22:07:49.976701 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 22:07:49.977210 4745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:49.966911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42887 (local address 127.1.244.194:33461)
0923 22:07:49.967143 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:49.967182 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:49.967738 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:49.967972 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:49.967975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:49.968551 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:49.968759 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:49.968764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.969494 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.969495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:49.970376 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:49.970379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:49.970971 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:49.970972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:49.971132 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:49.971785 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:49.971794 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:49.973191 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:49.975220 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:49.975228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:49.975231 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:49.975607 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:49.975865 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:49.975866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:49.975866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:49.976016 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:49.976310 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:49.976317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:49.976469 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 22:07:49.976575 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:07:49.976751 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":161,"threads_started":1}
I20260923 22:07:49.978377 4743 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42887
I20260923 22:07:49.978507 4743 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:49.978717 4743 heartbeater.cc:511] Master 127.1.244.254:42887 requested a full tablet report, sending...
I20260923 22:07:49.979146 4418 ts_manager.cc:194] Registered new tserver with Master: 0e4b6279f07f4adbab347372def770e0 (127.1.244.194:40193)
I20260923 22:07:49.979590 4418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:33461
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:07:49.999500 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42887
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:50.080816 4748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:50.081058 4748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:50.081099 4748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:50.082275 4748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:50.082343 4748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:50.082388 4748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:07:50.083956 4748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:50.084215 4748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:50.084448 4748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:50.089447 4756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:50.089567 4754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:50.089468 4753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:50.089677 4748 server_base.cc:1034] running on GCE node
I20260923 22:07:50.089893 4748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:50.090224 4748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:50.091388 4748 hybrid_clock.cc:648] HybridClock initialized: now 1790201270091370 us; error 31 us; skew 500 ppm
Sep 23 22:07:50 dist-test-slave-b5jz krb5kdc[4373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201270, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:50.093957 4748 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:07:50.094400 4748 webserver.cc:460] Webserver started at http://127.1.244.195:43479/ using document root <none> and password file <none>
I20260923 22:07:50.094646 4748 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:50.094715 4748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:50.094858 4748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:50.096004 4748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "ba7c37304fe2459eb19eedff9ed4bb54"
format_stamp: "Formatted at 2026-09-23 22:07:50 on dist-test-slave-b5jz"
server_key: "6633577d6b644d8ccd3905596ed54ac5"
server_key_iv: "cc1b6bc027185f4ef6e1a322d6d7dff1"
server_key_version: "encryptionkey@0"
I20260923 22:07:50.096359 4748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "ba7c37304fe2459eb19eedff9ed4bb54"
format_stamp: "Formatted at 2026-09-23 22:07:50 on dist-test-slave-b5jz"
server_key: "6633577d6b644d8ccd3905596ed54ac5"
server_key_iv: "cc1b6bc027185f4ef6e1a322d6d7dff1"
server_key_version: "encryptionkey@0"
I20260923 22:07:50.097651 4748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:50.098379 4763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:50.098577 4748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:50.098683 4748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "ba7c37304fe2459eb19eedff9ed4bb54"
format_stamp: "Formatted at 2026-09-23 22:07:50 on dist-test-slave-b5jz"
server_key: "6633577d6b644d8ccd3905596ed54ac5"
server_key_iv: "cc1b6bc027185f4ef6e1a322d6d7dff1"
server_key_version: "encryptionkey@0"
I20260923 22:07:50.098755 4748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:50.107175 4748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:50.109495 4748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:50.109658 4748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:50.109925 4748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:50.110229 4748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:50.110285 4748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:50.110327 4748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:50.110355 4748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:50.116449 4748 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:37857
I20260923 22:07:50.116489 4876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:37857 every 8 connection(s)
I20260923 22:07:50.116825 4748 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:07:50 dist-test-slave-b5jz krb5kdc[4373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201270, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:50.124491 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 4748
I20260923 22:07:50.124601 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:50.124792 2003 external_mini_cluster.cc:1428] Setting key 4c197d57414e67a6e7132f7344ff60ef
I20260923 22:07:50.126041 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:50.117605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:54741 (local address 127.1.244.254:42887)
0923 22:07:50.117675 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:50.117677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:50.117871 (+ 194us) server_negotiation.cc:407] Connection header received
0923 22:07:50.118331 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:50.118332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:50.118344 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:50.118397 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:50.119310 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:50.119867 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:50.120852 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:50.121029 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:50.121078 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:50.123186 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:50.123187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:50.123188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:50.123195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:50.124487 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:50.124979 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:50.124980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:50.124981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:50.125046 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:50.125367 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:50.125368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:50.125368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:50.125516 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:50.125583 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:50.125912 (+ 329us) server_negotiation.cc:299] Negotiation successful
0923 22:07:50.125980 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:07:50.126277 4879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:50.117593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42887 (local address 127.1.244.195:54741)
0923 22:07:50.117747 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:50.117783 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:50.118249 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:50.118451 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:50.118455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:50.119039 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:50.119234 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:50.119239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:50.119970 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:50.119971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:50.120778 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:50.120781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:50.121098 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:50.121099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:50.121195 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:50.121754 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:50.121763 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:50.123102 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:50.124566 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:50.124573 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:50.124575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:50.124892 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:50.125124 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:50.125125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:50.125125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:50.125286 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:50.125590 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:50.125595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:50.125772 (+ 177us) client_negotiation.cc:769] Sending connection context
0923 22:07:50.125886 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 22:07:50.126012 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260923 22:07:50.126739 4877 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42887
I20260923 22:07:50.126852 4877 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:50.127030 4877 heartbeater.cc:511] Master 127.1.244.254:42887 requested a full tablet report, sending...
I20260923 22:07:50.127429 4418 ts_manager.cc:194] Registered new tserver with Master: ba7c37304fe2459eb19eedff9ed4bb54 (127.1.244.195:37857)
I20260923 22:07:50.127832 4418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:54741
I20260923 22:07:50.136612 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:07:50.147619 4460 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:07:50.146591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55856 (local address 127.1.244.254:42887)
0923 22:07:50.146678 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:50.146680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:50.146766 (+ 86us) server_negotiation.cc:407] Connection header received
0923 22:07:50.146868 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:50.146869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:50.146880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:50.146941 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:50.147509 (+ 568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55856: BlockingRecv error: recv got EOF from 127.0.0.1:55856 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 22:07:50.148590 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 4480
I20260923 22:07:50.153381 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 4614
I20260923 22:07:50.157734 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 4748
I20260923 22:07:50.162086 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 4386
2026-09-23T22:07:50Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1891 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:50 dist-test-slave-b5jz krb5kdc[4889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:50 dist-test-slave-b5jz krb5kdc[4889](info): set up 2 sockets
Sep 23 22:07:50 dist-test-slave-b5jz krb5kdc[4889](info): commencing operation
krb5kdc: starting...
W20260923 22:07:51.191381 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:07:51 dist-test-slave-b5jz krb5kdc[4889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:07:51.305915 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46281
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:51.385931 4902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:51.386139 4902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:51.386161 4902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:51.387354 4902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:51.387398 4902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:51.387413 4902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:51.387425 4902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:51.387437 4902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:51.388971 4902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:51.389246 4902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:51.389475 4902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:51.394311 4910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:51.394339 4908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:51.394340 4907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:51.394616 4902 server_base.cc:1034] running on GCE node
I20260923 22:07:51.394884 4902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:51.395217 4902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:51.396378 4902 hybrid_clock.cc:648] HybridClock initialized: now 1790201271396374 us; error 39 us; skew 500 ppm
Sep 23 22:07:51 dist-test-slave-b5jz krb5kdc[4889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201271, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:51.398785 4902 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:07:51.399307 4902 webserver.cc:460] Webserver started at http://127.1.244.254:37533/ using document root <none> and password file <none>
I20260923 22:07:51.399580 4902 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:51.399643 4902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:51.399782 4902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:51.400839 4902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "e2a75dd0e33d4296a3a712355f6afde8"
format_stamp: "Formatted at 2026-09-23 22:07:51 on dist-test-slave-b5jz"
server_key: "c848b7f3701e0684f9f7c837412ad8ce"
server_key_iv: "49057060d9450c3428112ff3b61d1647"
server_key_version: "encryptionkey@0"
I20260923 22:07:51.401212 4902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "e2a75dd0e33d4296a3a712355f6afde8"
format_stamp: "Formatted at 2026-09-23 22:07:51 on dist-test-slave-b5jz"
server_key: "c848b7f3701e0684f9f7c837412ad8ce"
server_key_iv: "49057060d9450c3428112ff3b61d1647"
server_key_version: "encryptionkey@0"
I20260923 22:07:51.402519 4902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:07:51.403201 4917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:51.403378 4902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:51.403478 4902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "e2a75dd0e33d4296a3a712355f6afde8"
format_stamp: "Formatted at 2026-09-23 22:07:51 on dist-test-slave-b5jz"
server_key: "c848b7f3701e0684f9f7c837412ad8ce"
server_key_iv: "49057060d9450c3428112ff3b61d1647"
server_key_version: "encryptionkey@0"
I20260923 22:07:51.403589 4902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:51.427104 4902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:51.429101 4902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:51.429276 4902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:51.433319 4902 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46281
I20260923 22:07:51.433357 4969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46281 every 8 connection(s)
I20260923 22:07:51.433724 4902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:51.434559 4970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:51.436910 4970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8: Bootstrap starting.
I20260923 22:07:51.437530 4970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:51.437831 4970 log.cc:824] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:51.438493 4970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8: No bootstrap required, opened a new log
I20260923 22:07:51.440052 4970 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a75dd0e33d4296a3a712355f6afde8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46281 } }
I20260923 22:07:51.440204 4970 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:51.440229 4970 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a75dd0e33d4296a3a712355f6afde8, State: Initialized, Role: FOLLOWER
I20260923 22:07:51.440434 4970 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a75dd0e33d4296a3a712355f6afde8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46281 } }
I20260923 22:07:51.440503 4970 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:51.440526 4970 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:51.440596 4970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:51.440608 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 4902
I20260923 22:07:51.440706 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:07:51.440899 2003 external_mini_cluster.cc:1428] Setting key e2629dd95a342caed3dde21d6b00f2e4
I20260923 22:07:51.441685 4970 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a75dd0e33d4296a3a712355f6afde8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46281 } }
I20260923 22:07:51.441787 4970 leader_election.cc:302] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2a75dd0e33d4296a3a712355f6afde8; no voters:
I20260923 22:07:51.441968 4970 leader_election.cc:288] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:51.442003 4975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:51.442211 4975 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [term 1 LEADER]: Becoming Leader. State: Replica: e2a75dd0e33d4296a3a712355f6afde8, State: Running, Role: LEADER
I20260923 22:07:51.442299 4975 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a75dd0e33d4296a3a712355f6afde8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46281 } }
I20260923 22:07:51.442332 4970 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:51.442611 4975 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2a75dd0e33d4296a3a712355f6afde8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a75dd0e33d4296a3a712355f6afde8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46281 } } }
I20260923 22:07:51.442692 4975 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:51.442821 4975 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2a75dd0e33d4296a3a712355f6afde8. Latest consensus state: current_term: 1 leader_uuid: "e2a75dd0e33d4296a3a712355f6afde8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a75dd0e33d4296a3a712355f6afde8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46281 } } }
I20260923 22:07:51.442891 4975 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:51.443668 4981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:51.444298 4981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:51.445820 4981 catalog_manager.cc:1347] Generated new cluster ID: 556d8bbea198450cb65822e3ab488bbb
I20260923 22:07:51.445874 4981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:07:51 dist-test-slave-b5jz krb5kdc[4889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:51.452700 4974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:51.441472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.1.244.254:46281)
0923 22:07:51.441671 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:51.441678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:51.441704 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:07:51.441833 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:51.441840 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:51.441911 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:51.442114 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:51.442900 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:51.444349 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:51.445446 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:51.446068 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:51.446172 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:51.448988 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:51.448993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:51.448993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:51.449005 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:51.451000 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:51.451384 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:51.451388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:51.451389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:51.451532 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:51.451773 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:51.451774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:51.451775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:51.452098 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:51.452274 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:51.452346 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:07:51.452484 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
I20260923 22:07:51.453702 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:07:51.462637 4981 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:51.463325 4981 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:51.467614 4981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2a75dd0e33d4296a3a712355f6afde8: Generated new TSK 0
I20260923 22:07:51.467800 4981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:07:51.475235 4974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:51.471214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.1.244.254:46281)
0923 22:07:51.471270 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:51.471272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:51.471427 (+ 155us) server_negotiation.cc:407] Connection header received
0923 22:07:51.471462 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:51.471463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:51.471475 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:51.471611 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:51.471963 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:51.472756 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:51.473556 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:51.473899 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:51.473972 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:51.474670 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:51.474671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:51.474671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:07:51.474902 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:51.474920 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:07:51.474928 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:51.474991 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:51.475111 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 22:07:51.475138 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:07:51.477783 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 4902
2026-09-23T22:07:51Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1314 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:51 dist-test-slave-b5jz krb5kdc[5003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:51 dist-test-slave-b5jz krb5kdc[5003](info): set up 2 sockets
Sep 23 22:07:51 dist-test-slave-b5jz krb5kdc[5003](info): commencing operation
krb5kdc: starting...
W20260923 22:07:52.519536 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:07:52 dist-test-slave-b5jz krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:07:52.638139 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42057
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:42057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:52.720715 5016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:52.720968 5016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:52.721015 5016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:52.722178 5016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:52.722250 5016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:52.722275 5016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:52.722306 5016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:52.722335 5016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:52.723819 5016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:42057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:52.724100 5016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:52.724325 5016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:52.729056 5024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:52.729056 5022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:52.729204 5016 server_base.cc:1034] running on GCE node
W20260923 22:07:52.729326 5021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:52.729590 5016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:52.729900 5016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:52.731062 5016 hybrid_clock.cc:648] HybridClock initialized: now 1790201272731051 us; error 44 us; skew 500 ppm
Sep 23 22:07:52 dist-test-slave-b5jz krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201272, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:52.734337 5016 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:07:52.734957 5016 webserver.cc:460] Webserver started at http://127.1.244.254:46525/ using document root <none> and password file <none>
I20260923 22:07:52.735337 5016 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:52.735425 5016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:52.735589 5016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:52.736654 5016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "d22b5ac88c0f4e0898708f82b61bc709"
format_stamp: "Formatted at 2026-09-23 22:07:52 on dist-test-slave-b5jz"
server_key: "b723ea0b0fee1982b3490cb3d126d799"
server_key_iv: "1be437fec2c76c7f932909d51ea446fd"
server_key_version: "encryptionkey@0"
I20260923 22:07:52.737056 5016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "d22b5ac88c0f4e0898708f82b61bc709"
format_stamp: "Formatted at 2026-09-23 22:07:52 on dist-test-slave-b5jz"
server_key: "b723ea0b0fee1982b3490cb3d126d799"
server_key_iv: "1be437fec2c76c7f932909d51ea446fd"
server_key_version: "encryptionkey@0"
I20260923 22:07:52.738950 5016 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260923 22:07:52.740007 5031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:52.740186 5016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:52.740288 5016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "d22b5ac88c0f4e0898708f82b61bc709"
format_stamp: "Formatted at 2026-09-23 22:07:52 on dist-test-slave-b5jz"
server_key: "b723ea0b0fee1982b3490cb3d126d799"
server_key_iv: "1be437fec2c76c7f932909d51ea446fd"
server_key_version: "encryptionkey@0"
I20260923 22:07:52.740383 5016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:52.758775 5016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:52.761863 5016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:52.762017 5016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:52.767263 5016 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:42057
I20260923 22:07:52.767319 5083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:42057 every 8 connection(s)
I20260923 22:07:52.767776 5016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:52.769084 5084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:52.772899 5084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709: Bootstrap starting.
I20260923 22:07:52.773017 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 5016
I20260923 22:07:52.773124 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:07:52.773306 2003 external_mini_cluster.cc:1428] Setting key 9d09c02125c433a899632699fb0cfdb3
I20260923 22:07:52.773754 5084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:52.774190 5084 log.cc:824] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:52.775027 5084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709: No bootstrap required, opened a new log
I20260923 22:07:52.778187 5084 raft_consensus.cc:348] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22b5ac88c0f4e0898708f82b61bc709" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42057 } }
I20260923 22:07:52.778434 5084 raft_consensus.cc:374] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:52.778496 5084 raft_consensus.cc:729] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22b5ac88c0f4e0898708f82b61bc709, State: Initialized, Role: FOLLOWER
I20260923 22:07:52.778645 5084 consensus_queue.cc:260] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22b5ac88c0f4e0898708f82b61bc709" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42057 } }
I20260923 22:07:52.778762 5084 raft_consensus.cc:388] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:52.778801 5084 raft_consensus.cc:482] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:52.778856 5084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:52.779711 5084 raft_consensus.cc:504] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22b5ac88c0f4e0898708f82b61bc709" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42057 } }
I20260923 22:07:52.779841 5084 leader_election.cc:302] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d22b5ac88c0f4e0898708f82b61bc709; no voters:
I20260923 22:07:52.780057 5084 leader_election.cc:288] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:52.780076 5089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:52.780197 5089 raft_consensus.cc:686] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [term 1 LEADER]: Becoming Leader. State: Replica: d22b5ac88c0f4e0898708f82b61bc709, State: Running, Role: LEADER
I20260923 22:07:52.780280 5089 consensus_queue.cc:237] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22b5ac88c0f4e0898708f82b61bc709" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42057 } }
I20260923 22:07:52.780328 5084 sys_catalog.cc:564] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:52.780586 5089 sys_catalog.cc:455] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d22b5ac88c0f4e0898708f82b61bc709. Latest consensus state: current_term: 1 leader_uuid: "d22b5ac88c0f4e0898708f82b61bc709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22b5ac88c0f4e0898708f82b61bc709" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42057 } } }
I20260923 22:07:52.780669 5089 sys_catalog.cc:458] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:52.780588 5091 sys_catalog.cc:455] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d22b5ac88c0f4e0898708f82b61bc709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22b5ac88c0f4e0898708f82b61bc709" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42057 } } }
I20260923 22:07:52.780862 5091 sys_catalog.cc:458] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:52.780882 5094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:52.781517 5094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:07:52 dist-test-slave-b5jz krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:52.783658 5094 catalog_manager.cc:1347] Generated new cluster ID: 07991cb501204191a579d077bd7f2578
I20260923 22:07:52.783715 5094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:07:52.786332 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:52.773831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.1.244.254:42057)
0923 22:07:52.773999 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:52.774008 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:52.774046 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:07:52.774630 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:52.774640 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:52.774717 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:52.774868 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:52.775601 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:52.777019 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:52.777785 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:52.778205 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:52.778326 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:52.782400 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:52.782405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:52.782405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:52.782419 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:52.784287 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:52.784682 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:52.784685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:52.784686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:52.784786 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:52.785011 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:52.785012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:52.785013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:52.785313 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:52.785452 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:52.785984 (+ 532us) server_negotiation.cc:299] Negotiation successful
0923 22:07:52.786126 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260923 22:07:52.797940 5094 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:52.798461 5094 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:52.805253 5094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d22b5ac88c0f4e0898708f82b61bc709: Generated new TSK 0
I20260923 22:07:52.805444 5094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:07:52.834280 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42057
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:52.918112 5110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:52.918309 5110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:52.918332 5110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:52.919469 5110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:52.919512 5110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:52.919538 5110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:07:52.920991 5110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:52.921170 5110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:52.921337 5110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:52.926131 5116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:52.926149 5118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:52.926131 5115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:52.926422 5110 server_base.cc:1034] running on GCE node
I20260923 22:07:52.926640 5110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:52.926892 5110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:52.928048 5110 hybrid_clock.cc:648] HybridClock initialized: now 1790201272928043 us; error 35 us; skew 500 ppm
Sep 23 22:07:52 dist-test-slave-b5jz krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201272, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:52.930476 5110 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:07:52.930920 5110 webserver.cc:460] Webserver started at http://127.1.244.193:42643/ using document root <none> and password file <none>
I20260923 22:07:52.931106 5110 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:52.931145 5110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:52.931236 5110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:52.932257 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "e6de7043365c40e2945f9da296992a2b"
format_stamp: "Formatted at 2026-09-23 22:07:52 on dist-test-slave-b5jz"
server_key: "fb1213176ff8e03c579f64a8369d276c"
server_key_iv: "2feb713e8b8dc6d2e5e29ebfdc79c7d7"
server_key_version: "encryptionkey@0"
I20260923 22:07:52.932547 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "e6de7043365c40e2945f9da296992a2b"
format_stamp: "Formatted at 2026-09-23 22:07:52 on dist-test-slave-b5jz"
server_key: "fb1213176ff8e03c579f64a8369d276c"
server_key_iv: "2feb713e8b8dc6d2e5e29ebfdc79c7d7"
server_key_version: "encryptionkey@0"
I20260923 22:07:52.933746 5110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:52.934368 5125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:52.934540 5110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:52.934620 5110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "e6de7043365c40e2945f9da296992a2b"
format_stamp: "Formatted at 2026-09-23 22:07:52 on dist-test-slave-b5jz"
server_key: "fb1213176ff8e03c579f64a8369d276c"
server_key_iv: "2feb713e8b8dc6d2e5e29ebfdc79c7d7"
server_key_version: "encryptionkey@0"
I20260923 22:07:52.934674 5110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:52.946441 5110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:52.948294 5110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:52.948403 5110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:52.948577 5110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:52.948928 5110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:52.948963 5110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:52.949007 5110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:52.949033 5110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:52.954517 5110 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:42533
I20260923 22:07:52.954594 5238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:42533 every 8 connection(s)
I20260923 22:07:52.954910 5110 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:07:52.959107 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 5110
I20260923 22:07:52.959213 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:52.959427 2003 external_mini_cluster.cc:1428] Setting key d138393d45d2ca167db54e821cb70d46
Sep 23 22:07:52 dist-test-slave-b5jz krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201272, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:52.964635 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:52.955693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:37601 (local address 127.1.244.254:42057)
0923 22:07:52.955741 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:52.955743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:52.955960 (+ 217us) server_negotiation.cc:407] Connection header received
0923 22:07:52.956439 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:52.956440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:52.956449 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:52.956502 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:52.957399 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:52.957890 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:52.958845 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:52.959172 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:52.959252 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:52.961634 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:52.961635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:52.961636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:52.961642 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:52.963084 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:52.963635 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:52.963636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:52.963636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:52.963689 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:52.963982 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:52.963983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:52.963983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:52.964089 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:52.964137 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:52.964494 (+ 357us) server_negotiation.cc:299] Negotiation successful
0923 22:07:52.964572 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:07:52.964949 5241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:52.955690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42057 (local address 127.1.244.193:37601)
0923 22:07:52.955829 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:52.955865 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:52.956353 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:52.956557 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:52.956560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:52.957130 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:52.957323 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:52.957328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:52.957967 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:52.957968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:52.958773 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:52.958775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:52.959314 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:52.959315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:52.959412 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:52.959986 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:52.959996 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:52.961535 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:52.963186 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:52.963193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:52.963196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:52.963515 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:52.963760 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:52.963761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:52.963761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:52.963910 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:52.964156 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:52.964162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:52.964343 (+ 181us) client_negotiation.cc:769] Sending connection context
0923 22:07:52.964465 (+ 122us) client_negotiation.cc:240] Negotiation successful
0923 22:07:52.964652 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
I20260923 22:07:52.965517 5239 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42057
I20260923 22:07:52.965633 5239 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:52.965837 5239 heartbeater.cc:511] Master 127.1.244.254:42057 requested a full tablet report, sending...
I20260923 22:07:52.966351 5048 ts_manager.cc:194] Registered new tserver with Master: e6de7043365c40e2945f9da296992a2b (127.1.244.193:42533)
I20260923 22:07:52.966956 5048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:37601
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:07:52.986132 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42057
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:53.067294 5244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:53.067518 5244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:53.067560 5244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:53.068737 5244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:53.068783 5244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:53.068811 5244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:07:53.070255 5244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:53.070489 5244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:53.070708 5244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:53.075681 5250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:53.075697 5249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:53.075730 5244 server_base.cc:1034] running on GCE node
W20260923 22:07:53.075697 5252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:53.076201 5244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:53.076443 5244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:53.077620 5244 hybrid_clock.cc:648] HybridClock initialized: now 1790201273077596 us; error 33 us; skew 500 ppm
Sep 23 22:07:53 dist-test-slave-b5jz krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201273, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:53.080596 5244 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:07:53.081127 5244 webserver.cc:460] Webserver started at http://127.1.244.194:35037/ using document root <none> and password file <none>
I20260923 22:07:53.081357 5244 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:53.081401 5244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:53.081565 5244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:53.082800 5244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "283a37a9d6eb4ed5932c24c0fcb8fec0"
format_stamp: "Formatted at 2026-09-23 22:07:53 on dist-test-slave-b5jz"
server_key: "1f54f362c90c62d215b885fe3ecdce01"
server_key_iv: "620b33e1a41b2f0cd4e28613ecc5f2cd"
server_key_version: "encryptionkey@0"
I20260923 22:07:53.083202 5244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "283a37a9d6eb4ed5932c24c0fcb8fec0"
format_stamp: "Formatted at 2026-09-23 22:07:53 on dist-test-slave-b5jz"
server_key: "1f54f362c90c62d215b885fe3ecdce01"
server_key_iv: "620b33e1a41b2f0cd4e28613ecc5f2cd"
server_key_version: "encryptionkey@0"
I20260923 22:07:53.084592 5244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:07:53.085316 5259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:53.085498 5244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:53.085597 5244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "283a37a9d6eb4ed5932c24c0fcb8fec0"
format_stamp: "Formatted at 2026-09-23 22:07:53 on dist-test-slave-b5jz"
server_key: "1f54f362c90c62d215b885fe3ecdce01"
server_key_iv: "620b33e1a41b2f0cd4e28613ecc5f2cd"
server_key_version: "encryptionkey@0"
I20260923 22:07:53.085682 5244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:53.104908 5244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:53.107311 5244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:53.107478 5244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:53.107762 5244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:53.108080 5244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:53.108135 5244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:53.108171 5244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:53.108201 5244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:53.113891 5244 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:45383
I20260923 22:07:53.113967 5372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:45383 every 8 connection(s)
I20260923 22:07:53.114308 5244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:07:53 dist-test-slave-b5jz krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201273, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:53.121592 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 5244
I20260923 22:07:53.121709 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:53.121912 2003 external_mini_cluster.cc:1428] Setting key 357ed948e32648f83f92afd414e7e42b
I20260923 22:07:53.123960 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:53.115087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:38701 (local address 127.1.244.254:42057)
0923 22:07:53.115162 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:53.115166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:53.115365 (+ 199us) server_negotiation.cc:407] Connection header received
0923 22:07:53.115962 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:53.115963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:53.115976 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:53.116029 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:53.117021 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.117535 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.118574 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.118771 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.118850 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:53.120956 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:53.120957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:53.120957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:53.120965 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:53.122464 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:53.122904 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:53.122905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:53.122906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:53.122968 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:53.123200 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:53.123202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:53.123202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:53.123352 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:53.123421 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:53.123718 (+ 297us) server_negotiation.cc:299] Negotiation successful
0923 22:07:53.123818 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:07:53.123998 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:53.115059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42057 (local address 127.1.244.194:38701)
0923 22:07:53.115236 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:53.115279 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:53.115854 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:53.116118 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:53.116125 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:53.116689 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:53.116882 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:53.116887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.117656 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.117657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:53.118472 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:53.118474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.118855 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.118856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:53.118993 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:53.119465 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:53.119474 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:53.120854 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:53.122578 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:53.122585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:53.122587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:53.122831 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:53.123039 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:53.123039 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:53.123039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:53.123138 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:53.123412 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:53.123416 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:53.123530 (+ 114us) client_negotiation.cc:769] Sending connection context
0923 22:07:53.123678 (+ 148us) client_negotiation.cc:240] Negotiation successful
0923 22:07:53.123811 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260923 22:07:53.124442 5373 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42057
I20260923 22:07:53.124558 5373 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:53.124766 5373 heartbeater.cc:511] Master 127.1.244.254:42057 requested a full tablet report, sending...
I20260923 22:07:53.125113 5048 ts_manager.cc:194] Registered new tserver with Master: 283a37a9d6eb4ed5932c24c0fcb8fec0 (127.1.244.194:45383)
I20260923 22:07:53.125487 5048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:38701
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:07:53.146759 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42057
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:53.228157 5378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:53.228371 5378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:53.228394 5378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:53.229558 5378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:53.229604 5378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:53.229631 5378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:07:53.231159 5378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:53.231412 5378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:53.231686 5378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:53.236629 5386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:53.236639 5383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:53.236629 5384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:53.236941 5378 server_base.cc:1034] running on GCE node
I20260923 22:07:53.237165 5378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:53.237401 5378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:53.238562 5378 hybrid_clock.cc:648] HybridClock initialized: now 1790201273238545 us; error 29 us; skew 500 ppm
Sep 23 22:07:53 dist-test-slave-b5jz krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201273, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:53.241150 5378 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:07:53.241577 5378 webserver.cc:460] Webserver started at http://127.1.244.195:46333/ using document root <none> and password file <none>
I20260923 22:07:53.241817 5378 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:53.241885 5378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:53.242025 5378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:53.243050 5378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "c7006bfbb74749c29f45ca8d4e3dda34"
format_stamp: "Formatted at 2026-09-23 22:07:53 on dist-test-slave-b5jz"
server_key: "9d225c8bae726fed4b06dba5f9780da5"
server_key_iv: "49f4736f4e6244a8481905f87cfa810d"
server_key_version: "encryptionkey@0"
I20260923 22:07:53.243404 5378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "c7006bfbb74749c29f45ca8d4e3dda34"
format_stamp: "Formatted at 2026-09-23 22:07:53 on dist-test-slave-b5jz"
server_key: "9d225c8bae726fed4b06dba5f9780da5"
server_key_iv: "49f4736f4e6244a8481905f87cfa810d"
server_key_version: "encryptionkey@0"
I20260923 22:07:53.244783 5378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:07:53.245481 5393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:53.245640 5378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:53.245743 5378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "c7006bfbb74749c29f45ca8d4e3dda34"
format_stamp: "Formatted at 2026-09-23 22:07:53 on dist-test-slave-b5jz"
server_key: "9d225c8bae726fed4b06dba5f9780da5"
server_key_iv: "49f4736f4e6244a8481905f87cfa810d"
server_key_version: "encryptionkey@0"
I20260923 22:07:53.245818 5378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:53.256004 5378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:53.258286 5378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:53.258446 5378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:53.258664 5378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:53.258962 5378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:53.259016 5378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:53.259048 5378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:53.259075 5378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:53.264909 5378 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:46479
I20260923 22:07:53.264950 5506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:46479 every 8 connection(s)
I20260923 22:07:53.265259 5378 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:07:53 dist-test-slave-b5jz krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201273, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:53.271636 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 5378
I20260923 22:07:53.271750 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:53.271946 2003 external_mini_cluster.cc:1428] Setting key b70876a1845845c7612cf18fd352278f
I20260923 22:07:53.274454 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:53.265907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:32913 (local address 127.1.244.254:42057)
0923 22:07:53.265964 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:53.265967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:53.266207 (+ 240us) server_negotiation.cc:407] Connection header received
0923 22:07:53.266698 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:53.266699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:53.266709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:53.266762 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:53.267783 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.268270 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.269234 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.269421 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.269478 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:53.271723 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:53.271725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:53.271726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:53.271735 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:53.273323 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:53.273683 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:53.273684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:53.273684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:53.273743 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:53.273952 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:53.273953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:53.273953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:53.274049 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:53.274097 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:53.274337 (+ 240us) server_negotiation.cc:299] Negotiation successful
0923 22:07:53.274393 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:07:53.274672 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:53.265889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42057 (local address 127.1.244.195:32913)
0923 22:07:53.266069 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:53.266117 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:53.266608 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:53.266846 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:53.266850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:53.267420 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:53.267707 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:53.267713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.268348 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.268349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:53.269152 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:53.269154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.269495 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.269496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:53.269597 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:53.270120 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:53.270128 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:53.271610 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:53.273395 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:53.273400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:53.273401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:53.273617 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:53.273800 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:53.273801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:53.273801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:53.273896 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:53.274102 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:53.274106 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:53.274227 (+ 121us) client_negotiation.cc:769] Sending connection context
0923 22:07:53.274309 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 22:07:53.274425 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260923 22:07:53.275121 5507 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42057
I20260923 22:07:53.275202 5507 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:53.275389 5507 heartbeater.cc:511] Master 127.1.244.254:42057 requested a full tablet report, sending...
I20260923 22:07:53.275810 5048 ts_manager.cc:194] Registered new tserver with Master: c7006bfbb74749c29f45ca8d4e3dda34 (127.1.244.195:46479)
I20260923 22:07:53.276183 5048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:32913
I20260923 22:07:53.283778 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:07:53 dist-test-slave-b5jz krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201273, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 23 22:07:53 dist-test-slave-b5jz krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201273, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:53.306314 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:53.299115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.1.244.254:42057)
0923 22:07:53.299209 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:53.299211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:53.299308 (+ 97us) server_negotiation.cc:407] Connection header received
0923 22:07:53.299416 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:53.299417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:53.299430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:53.299495 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:53.300101 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.300607 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.301348 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:53.301546 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:53.301621 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:53.303893 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:53.303894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:53.303895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:53.303903 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:53.305076 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:53.305444 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:53.305445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:53.305446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:53.305505 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:53.305707 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:53.305708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:53.305708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:53.305892 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:53.305982 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:53.306091 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 22:07:53.306157 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260923 22:07:53.306551 5048 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43286
I20260923 22:07:53.306977 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 5110
I20260923 22:07:53.312125 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 5244
I20260923 22:07:53.316586 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 5378
I20260923 22:07:53.320623 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 5016
2026-09-23T22:07:53Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1844 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:53 dist-test-slave-b5jz krb5kdc[5519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:53 dist-test-slave-b5jz krb5kdc[5519](info): set up 2 sockets
Sep 23 22:07:53 dist-test-slave-b5jz krb5kdc[5519](info): commencing operation
krb5kdc: starting...
W20260923 22:07:54.401227 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260923 22:07:54.532873 2003 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.132s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260923 22:07:54.742964 2003 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.210s user 0.000s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260923 22:07:54.996470 2003 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.253s user 0.000s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/test-user.keytab.
W20260923 22:07:55.788985 2003 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.792s user 0.000s sys 0.002s
Sep 23 22:07:55 dist-test-slave-b5jz krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
W20260923 22:07:57.078408 2003 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.244.254: real 1.268s user 0.001s sys 0.002s
I20260923 22:07:57.078811 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35941
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:57.160336 5532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:57.160549 5532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:57.160573 5532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:57.161752 5532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:57.161795 5532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:57.161810 5532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:57.161861 5532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:57.161887 5532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:57.163404 5532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:57.163695 5532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:57.163939 5532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:57.168753 5540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:57.168900 5532 server_base.cc:1034] running on GCE node
W20260923 22:07:57.168753 5537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.168787 5538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:57.169318 5532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:57.169613 5532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:57.170783 5532 hybrid_clock.cc:648] HybridClock initialized: now 1790201277170770 us; error 30 us; skew 500 ppm
Sep 23 22:07:57 dist-test-slave-b5jz krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:57.173364 5532 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:07:57.173803 5532 webserver.cc:460] Webserver started at http://127.1.244.254:37133/ using document root <none> and password file <none>
I20260923 22:07:57.174042 5532 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:57.174120 5532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:57.174280 5532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:57.175345 5532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "8adc3523716547beab37effcae32fae0"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-b5jz"
server_key: "4b3ee66d6ce257c40b03f327832d0a1a"
server_key_iv: "f66712963c0e8d15ccb1ff0fae267bbf"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.175729 5532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "8adc3523716547beab37effcae32fae0"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-b5jz"
server_key: "4b3ee66d6ce257c40b03f327832d0a1a"
server_key_iv: "f66712963c0e8d15ccb1ff0fae267bbf"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.177080 5532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:07:57.177944 5547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.178161 5532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:07:57.178277 5532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "8adc3523716547beab37effcae32fae0"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-b5jz"
server_key: "4b3ee66d6ce257c40b03f327832d0a1a"
server_key_iv: "f66712963c0e8d15ccb1ff0fae267bbf"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.179020 5532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:57.196811 5532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:57.199154 5532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:57.199276 5532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:57.203127 5532 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35941
I20260923 22:07:57.203188 5599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35941 every 8 connection(s)
I20260923 22:07:57.203696 5532 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:57.203898 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 5532
I20260923 22:07:57.204022 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:07:57.204232 2003 external_mini_cluster.cc:1428] Setting key 6114cc4746c87dee2129d90da9072030
I20260923 22:07:57.204494 5600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:57.207219 5600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0: Bootstrap starting.
I20260923 22:07:57.207890 5600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:57.208160 5600 log.cc:824] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:57.208901 5600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0: No bootstrap required, opened a new log
Sep 23 22:07:57 dist-test-slave-b5jz krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:57.210886 5600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8adc3523716547beab37effcae32fae0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35941 } }
I20260923 22:07:57.211045 5600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:57.211098 5600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8adc3523716547beab37effcae32fae0, State: Initialized, Role: FOLLOWER
I20260923 22:07:57.211229 5600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8adc3523716547beab37effcae32fae0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35941 } }
I20260923 22:07:57.211326 5600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:57.211359 5600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:57.211408 5600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:57.215023 5603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.204714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58034 (local address 127.1.244.254:35941)
0923 22:07:57.204913 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.204922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.204960 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:07:57.205602 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.205610 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.205680 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.205833 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.206469 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.207466 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.208363 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.208909 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.209028 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.211605 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:57.211609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:57.211609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:57.211620 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:57.213763 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.214100 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.214102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.214102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.214182 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.214340 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.214340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.214341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.214576 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:57.214679 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.214785 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.214881 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":88,"threads_started":1}
I20260923 22:07:57.249863 5600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8adc3523716547beab37effcae32fae0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35941 } }
I20260923 22:07:57.250061 5600 leader_election.cc:302] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8adc3523716547beab37effcae32fae0; no voters:
I20260923 22:07:57.250329 5600 leader_election.cc:288] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:57.250411 5605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:57.250609 5605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [term 1 LEADER]: Becoming Leader. State: Replica: 8adc3523716547beab37effcae32fae0, State: Running, Role: LEADER
I20260923 22:07:57.250628 5600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:57.250785 5605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8adc3523716547beab37effcae32fae0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35941 } }
I20260923 22:07:57.251313 5607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8adc3523716547beab37effcae32fae0. Latest consensus state: current_term: 1 leader_uuid: "8adc3523716547beab37effcae32fae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8adc3523716547beab37effcae32fae0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35941 } } }
I20260923 22:07:57.251504 5607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:57.251746 5606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8adc3523716547beab37effcae32fae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8adc3523716547beab37effcae32fae0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35941 } } }
I20260923 22:07:57.251885 5606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:57.252184 5614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:57.252808 5614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:57.254498 5614 catalog_manager.cc:1347] Generated new cluster ID: 8d900063756247cc934c7ae63ade03b8
I20260923 22:07:57.254551 5614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:07:57.271502 5614 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:57.272305 5614 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:57.281857 5614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8adc3523716547beab37effcae32fae0: Generated new TSK 0
I20260923 22:07:57.282073 5614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
W20260923 22:07:57.701848 2003 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.244.193: real 0.413s user 0.001s sys 0.002s
I20260923 22:07:57.702198 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35941
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:57.782548 5626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:57.782820 5626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:57.782846 5626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:57.784052 5626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:57.784092 5626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:57.784140 5626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:07:57.785655 5626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:57.785895 5626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:57.786113 5626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:07:57.790911 5626 server_base.cc:1034] running on GCE node
W20260923 22:07:57.790886 5632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.790921 5631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.790895 5634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:57.791270 5626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:57.791483 5626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:57.792630 5626 hybrid_clock.cc:648] HybridClock initialized: now 1790201277792621 us; error 30 us; skew 500 ppm
Sep 23 22:07:57 dist-test-slave-b5jz krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:57.795149 5626 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:07:57.795626 5626 webserver.cc:460] Webserver started at http://127.1.244.193:42559/ using document root <none> and password file <none>
I20260923 22:07:57.795823 5626 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:57.795864 5626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:57.795949 5626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:57.796908 5626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "a38e656af264457ea90ed6dfc0a9048e"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-b5jz"
server_key: "43340a9609fcaf5def9ef0ec9893dd95"
server_key_iv: "bb93bbd8c207f5d26126812edbeda4a0"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.797206 5626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "a38e656af264457ea90ed6dfc0a9048e"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-b5jz"
server_key: "43340a9609fcaf5def9ef0ec9893dd95"
server_key_iv: "bb93bbd8c207f5d26126812edbeda4a0"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.798404 5626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:07:57.799077 5641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.799243 5626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:57.799341 5626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "a38e656af264457ea90ed6dfc0a9048e"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-b5jz"
server_key: "43340a9609fcaf5def9ef0ec9893dd95"
server_key_iv: "bb93bbd8c207f5d26126812edbeda4a0"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.799419 5626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:57.819093 5626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:57.821079 5626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:57.821251 5626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:57.821489 5626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:57.821805 5626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:57.821858 5626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.821899 5626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:57.821926 5626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.827751 5626 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:34781
I20260923 22:07:57.827809 5754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:34781 every 8 connection(s)
I20260923 22:07:57.828141 5626 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:07:57 dist-test-slave-b5jz krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:57.836851 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 5626
I20260923 22:07:57.836956 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:57.837162 2003 external_mini_cluster.cc:1428] Setting key 691e20bc23d68577c5b4dac6b2b9f7bf
I20260923 22:07:57.837653 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.828861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:49443 (local address 127.1.244.254:35941)
0923 22:07:57.828998 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.829000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.829150 (+ 150us) server_negotiation.cc:407] Connection header received
0923 22:07:57.830153 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.830155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.830170 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.830227 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.830885 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.831398 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.832472 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.832695 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.832788 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.834810 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:57.834811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:57.834812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:57.834819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:57.836162 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.836587 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.836589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.836589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.836650 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.836908 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.836910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.836910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.837068 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:57.837128 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.837512 (+ 384us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.837587 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":56,"threads_started":1}
I20260923 22:07:57.837828 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.828828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35941 (local address 127.1.244.193:49443)
0923 22:07:57.829008 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.829059 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.830017 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.830326 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.830329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.830582 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:57.830765 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.830769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.831496 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.831498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.832372 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.832375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.832798 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.832799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.832935 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.833388 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:57.833396 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:57.834696 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:57.836269 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:57.836276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:57.836278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:57.836495 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:57.836731 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:57.836732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:57.836732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:57.836839 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:57.837128 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:57.837136 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:57.837309 (+ 173us) client_negotiation.cc:769] Sending connection context
0923 22:07:57.837429 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 22:07:57.837588 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260923 22:07:57.838366 5755 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35941
I20260923 22:07:57.838460 5755 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:57.838651 5755 heartbeater.cc:511] Master 127.1.244.254:35941 requested a full tablet report, sending...
I20260923 22:07:57.839188 5564 ts_manager.cc:194] Registered new tserver with Master: a38e656af264457ea90ed6dfc0a9048e (127.1.244.193:34781)
I20260923 22:07:57.839874 5564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:49443
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:07:57.861690 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35941
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:57.941802 5761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:57.942025 5761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:57.942061 5761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:57.943179 5761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:57.943246 5761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:57.943289 5761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:07:57.944795 5761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:57.945070 5761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:57.945289 5761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:57.950109 5769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.950098 5767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.950109 5766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:57.950469 5761 server_base.cc:1034] running on GCE node
I20260923 22:07:57.950726 5761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:57.950964 5761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:57.952105 5761 hybrid_clock.cc:648] HybridClock initialized: now 1790201277952085 us; error 36 us; skew 500 ppm
Sep 23 22:07:57 dist-test-slave-b5jz krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:57.954497 5761 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:07:57.954957 5761 webserver.cc:460] Webserver started at http://127.1.244.194:35103/ using document root <none> and password file <none>
I20260923 22:07:57.955199 5761 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:57.955274 5761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:57.955430 5761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:57.956476 5761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "e61795a0de20419f9225424c3aa45eb7"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-b5jz"
server_key: "d2fe4e614c0557f18de06ee8fd261839"
server_key_iv: "87d76c3cd29c63a510234792f7f6d1c2"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.956833 5761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "e61795a0de20419f9225424c3aa45eb7"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-b5jz"
server_key: "d2fe4e614c0557f18de06ee8fd261839"
server_key_iv: "87d76c3cd29c63a510234792f7f6d1c2"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.958168 5761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:07:57.958884 5776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.959050 5761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:57.959151 5761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "e61795a0de20419f9225424c3aa45eb7"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-b5jz"
server_key: "d2fe4e614c0557f18de06ee8fd261839"
server_key_iv: "87d76c3cd29c63a510234792f7f6d1c2"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.959236 5761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:57.986053 5761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:57.988193 5761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:57.988369 5761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:57.988618 5761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:57.988951 5761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:57.989007 5761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.989051 5761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:57.989075 5761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.994890 5761 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:46819
I20260923 22:07:57.994932 5889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:46819 every 8 connection(s)
I20260923 22:07:57.995304 5761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:07:57.996291 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 5761
I20260923 22:07:57.996402 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:57.996585 2003 external_mini_cluster.cc:1428] Setting key f8d4644b662f7ddba7ca44c2d70c3213
Sep 23 22:07:57 dist-test-slave-b5jz krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
I20260923 22:07:58.006743 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.996048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:60227 (local address 127.1.244.254:35941)
0923 22:07:57.996091 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.996093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.996302 (+ 209us) server_negotiation.cc:407] Connection header received
0923 22:07:57.997622 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.997623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.997638 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.997698 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.998402 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.999123 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.000623 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.000998 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.001079 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:58.003709 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:58.003711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:58.003711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:58.003717 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:58.005160 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.005671 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.005672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.005673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.005723 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.006011 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.006012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.006012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.006170 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:58.006253 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:58.006561 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 22:07:58.006649 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:07:58.006938 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.996046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35941 (local address 127.1.244.194:60227)
0923 22:07:57.996192 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.996227 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.997495 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.997758 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.997761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.998078 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:57.998297 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.998302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.999236 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.999238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:58.000518 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:58.000521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.001110 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.001112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:58.001243 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:58.001789 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:58.001797 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:58.003621 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:58.005250 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:58.005257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:58.005259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:58.005557 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:58.005793 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:58.005794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:58.005795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:58.005928 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:58.006251 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:58.006257 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:58.006410 (+ 153us) client_negotiation.cc:769] Sending connection context
0923 22:07:58.006540 (+ 130us) client_negotiation.cc:240] Negotiation successful
0923 22:07:58.006665 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260923 22:07:58.007696 5890 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35941
I20260923 22:07:58.007800 5890 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:58.008008 5890 heartbeater.cc:511] Master 127.1.244.254:35941 requested a full tablet report, sending...
I20260923 22:07:58.008613 5564 ts_manager.cc:194] Registered new tserver with Master: e61795a0de20419f9225424c3aa45eb7 (127.1.244.194:46819)
I20260923 22:07:58.009105 5564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:60227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:07:58.022694 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35941
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:58.100773 5895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:58.100966 5895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:58.100987 5895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:58.102208 5895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:58.102248 5895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:58.102382 5895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:07:58.104012 5895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:58.104245 5895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:58.104475 5895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:58.109190 5903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:58.109174 5900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:58.109174 5901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:58.109422 5895 server_base.cc:1034] running on GCE node
I20260923 22:07:58.109629 5895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:58.109814 5895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:58.110955 5895 hybrid_clock.cc:648] HybridClock initialized: now 1790201278110943 us; error 28 us; skew 500 ppm
Sep 23 22:07:58 dist-test-slave-b5jz krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201278, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:58.113335 5895 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:07:58.113783 5895 webserver.cc:460] Webserver started at http://127.1.244.195:41199/ using document root <none> and password file <none>
I20260923 22:07:58.113997 5895 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:58.114063 5895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:58.114205 5895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:58.115187 5895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "29a43fc4c5494ee98a246fa318325fb8"
format_stamp: "Formatted at 2026-09-23 22:07:58 on dist-test-slave-b5jz"
server_key: "ef35f716f64f40c639a8731531744e9b"
server_key_iv: "8b5845e17c4b7d22318c457336f4fd9d"
server_key_version: "encryptionkey@0"
I20260923 22:07:58.115538 5895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "29a43fc4c5494ee98a246fa318325fb8"
format_stamp: "Formatted at 2026-09-23 22:07:58 on dist-test-slave-b5jz"
server_key: "ef35f716f64f40c639a8731531744e9b"
server_key_iv: "8b5845e17c4b7d22318c457336f4fd9d"
server_key_version: "encryptionkey@0"
I20260923 22:07:58.116863 5895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:58.117569 5910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:58.117745 5895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:58.117846 5895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "29a43fc4c5494ee98a246fa318325fb8"
format_stamp: "Formatted at 2026-09-23 22:07:58 on dist-test-slave-b5jz"
server_key: "ef35f716f64f40c639a8731531744e9b"
server_key_iv: "8b5845e17c4b7d22318c457336f4fd9d"
server_key_version: "encryptionkey@0"
I20260923 22:07:58.117931 5895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:58.127090 5895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:58.128836 5895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:58.128993 5895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:58.129194 5895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:58.129480 5895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:58.129532 5895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:58.129566 5895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:58.129592 5895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:58.135176 5895 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:38097
I20260923 22:07:58.135221 6023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:38097 every 8 connection(s)
I20260923 22:07:58.135504 5895 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:07:58.136674 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 5895
I20260923 22:07:58.136783 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:58.136958 2003 external_mini_cluster.cc:1428] Setting key c51fdd3cdc656aec1382593f1b5e64b1
Sep 23 22:07:58 dist-test-slave-b5jz krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201278, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:58.144402 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:58.136152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:44391 (local address 127.1.244.254:35941)
0923 22:07:58.136232 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:58.136234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:58.136408 (+ 174us) server_negotiation.cc:407] Connection header received
0923 22:07:58.137542 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:58.137543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:58.137555 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:58.137602 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:58.138296 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.138766 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.139815 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.139991 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.140043 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:58.142072 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:58.142073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:58.142073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:58.142078 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:58.143243 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.143647 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.143647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.143648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.143696 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.143906 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.143907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.143907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.144001 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:58.144052 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:58.144273 (+ 221us) server_negotiation.cc:299] Negotiation successful
0923 22:07:58.144339 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 22:07:58.144542 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:58.136119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35941 (local address 127.1.244.195:44391)
0923 22:07:58.136267 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:58.136298 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:58.137444 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:58.137672 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:58.137676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:58.138028 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:58.138206 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:58.138210 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.138841 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.138842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:58.139744 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:58.139746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.140060 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.140060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:58.140198 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:58.140701 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:58.140709 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:58.141995 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:58.143324 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:58.143329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:58.143331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:58.143533 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:58.143756 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:58.143757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:58.143757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:58.143852 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:58.144053 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:58.144057 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:58.144160 (+ 103us) client_negotiation.cc:769] Sending connection context
0923 22:07:58.144248 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 22:07:58.144359 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260923 22:07:58.144989 6024 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35941
I20260923 22:07:58.145087 6024 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:58.145262 6024 heartbeater.cc:511] Master 127.1.244.254:35941 requested a full tablet report, sending...
I20260923 22:07:58.145615 5564 ts_manager.cc:194] Registered new tserver with Master: 29a43fc4c5494ee98a246fa318325fb8 (127.1.244.195:38097)
I20260923 22:07:58.145969 5564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:44391
I20260923 22:07:58.148628 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:07:58 dist-test-slave-b5jz krb5kdc[5519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:07:58 dist-test-slave-b5jz krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:07:58.170166 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:58.163071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54250 (local address 127.1.244.193:34781)
0923 22:07:58.163235 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:58.163240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:58.163250 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:07:58.163277 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:58.163278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:58.163292 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:58.163357 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:58.163948 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.164640 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.165333 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.165626 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.165677 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:58.167839 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:58.167846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:58.167846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:58.167852 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:58.169203 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.169498 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.169501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.169501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.169566 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.169708 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.169708 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.169708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.169824 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:58.169919 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:58.169985 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:07:58.170041 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
Sep 23 22:07:58 dist-test-slave-b5jz krb5kdc[5519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:58.177364 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:58.171330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.1.244.254:35941)
0923 22:07:58.171384 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:58.171386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:58.171395 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:07:58.171423 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:58.171424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:58.171432 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:58.171476 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:58.171965 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.172419 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.173011 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.173177 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.173223 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:58.175277 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:58.175278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:58.175278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:58.175283 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:58.176483 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.176771 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.176771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.176772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.176817 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.176957 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.176958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.176958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.177088 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:58.177159 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:58.177252 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 22:07:58.177305 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260923 22:07:58.177589 5564 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58048
I20260923 22:07:58.177914 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 5626
I20260923 22:07:58.182845 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 5761
I20260923 22:07:58.186861 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 5895
I20260923 22:07:58.190877 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 5532
2026-09-23T22:07:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4870 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:58 dist-test-slave-b5jz krb5kdc[6036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:58 dist-test-slave-b5jz krb5kdc[6036](info): set up 2 sockets
Sep 23 22:07:58 dist-test-slave-b5jz krb5kdc[6036](info): commencing operation
krb5kdc: starting...
W20260923 22:07:59.223429 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:07:59 dist-test-slave-b5jz krb5kdc[6036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:07:59.333972 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42811
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:42811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:59.416767 6049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:59.416960 6049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:59.416985 6049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:59.418212 6049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:59.418253 6049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:59.418267 6049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:59.418303 6049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:59.418332 6049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:59.419826 6049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:42811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:59.420077 6049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:59.420300 6049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:59.425150 6054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.425129 6057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.425161 6049 server_base.cc:1034] running on GCE node
W20260923 22:07:59.425125 6055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.425624 6049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:59.425870 6049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:59.427008 6049 hybrid_clock.cc:648] HybridClock initialized: now 1790201279426996 us; error 30 us; skew 500 ppm
Sep 23 22:07:59 dist-test-slave-b5jz krb5kdc[6036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:59.429368 6049 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:07:59.429687 6049 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:59.429734 6049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:59.429876 6049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:59.430882 6049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "b6d05884572d4d25a20583db1a42bef1"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "3bde9dcb0a74c5651170e0dc114c5fa0"
server_key_iv: "7472a63450f6d8d339cce1392e2ad77d"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.431222 6049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "b6d05884572d4d25a20583db1a42bef1"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "3bde9dcb0a74c5651170e0dc114c5fa0"
server_key_iv: "7472a63450f6d8d339cce1392e2ad77d"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.432547 6049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:07:59.433218 6063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.433394 6049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:59.433492 6049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "b6d05884572d4d25a20583db1a42bef1"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "3bde9dcb0a74c5651170e0dc114c5fa0"
server_key_iv: "7472a63450f6d8d339cce1392e2ad77d"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.433568 6049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:59.443065 6049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:59.444828 6049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:59.444984 6049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:59.448645 6049 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:42811
I20260923 22:07:59.448669 6115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:42811 every 8 connection(s)
I20260923 22:07:59.448935 6049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:59.449806 6116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.452011 6116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1: Bootstrap starting.
I20260923 22:07:59.452524 6116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.452765 6116 log.cc:824] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:59.453372 6116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1: No bootstrap required, opened a new log
I20260923 22:07:59.454844 6116 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d05884572d4d25a20583db1a42bef1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42811 } }
I20260923 22:07:59.454972 6116 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.454993 6116 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d05884572d4d25a20583db1a42bef1, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.455123 6116 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d05884572d4d25a20583db1a42bef1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42811 } }
I20260923 22:07:59.455199 6116 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:59.455222 6116 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:59.455251 6116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:59.456045 6116 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d05884572d4d25a20583db1a42bef1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42811 } }
I20260923 22:07:59.456130 6116 leader_election.cc:302] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6d05884572d4d25a20583db1a42bef1; no voters:
I20260923 22:07:59.456254 6116 leader_election.cc:288] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:59.456331 6119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:59.456483 6116 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:59.456492 6119 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [term 1 LEADER]: Becoming Leader. State: Replica: b6d05884572d4d25a20583db1a42bef1, State: Running, Role: LEADER
I20260923 22:07:59.456696 6119 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d05884572d4d25a20583db1a42bef1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42811 } }
I20260923 22:07:59.457137 6121 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6d05884572d4d25a20583db1a42bef1. Latest consensus state: current_term: 1 leader_uuid: "b6d05884572d4d25a20583db1a42bef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d05884572d4d25a20583db1a42bef1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42811 } } }
I20260923 22:07:59.457224 6121 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:59.457185 6120 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6d05884572d4d25a20583db1a42bef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d05884572d4d25a20583db1a42bef1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42811 } } }
I20260923 22:07:59.457283 6120 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:59.457427 6127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:59.457922 6127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:59.458613 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 6049
I20260923 22:07:59.458735 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:07:59.458925 2003 external_mini_cluster.cc:1428] Setting key 11f4b7e1205eef4f3b5acaf63b66758a
I20260923 22:07:59.459774 6127 catalog_manager.cc:1347] Generated new cluster ID: 159bcd5e8abc41d5a05cc27ac4ed78f8
I20260923 22:07:59.459823 6127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:07:59 dist-test-slave-b5jz krb5kdc[6036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:59.468950 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.459423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35006 (local address 127.1.244.254:42811)
0923 22:07:59.459583 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.459591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.459625 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:07:59.459810 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.459817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.459874 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.460020 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.460618 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.461681 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.462763 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.463204 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.463336 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.465616 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.465620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.465621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.465632 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.467313 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.467776 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.467779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.467779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.467874 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.468095 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.468096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.468096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.468334 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.468506 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.468586 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.468729 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260923 22:07:59.470489 6127 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:59.470979 6127 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:59.478421 6127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1: Generated new TSK 0
I20260923 22:07:59.478602 6127 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:07:59.506590 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42811
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:59.585937 6142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:59.586136 6142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:59.586160 6142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:59.587249 6142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:59.587282 6142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:59.587308 6142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:07:59.588747 6142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:59.588925 6142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:59.589089 6142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:59.593880 6148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.593875 6150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.593876 6147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.594249 6142 server_base.cc:1034] running on GCE node
I20260923 22:07:59.594470 6142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:59.594661 6142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:59.595798 6142 hybrid_clock.cc:648] HybridClock initialized: now 1790201279595781 us; error 33 us; skew 500 ppm
Sep 23 22:07:59 dist-test-slave-b5jz krb5kdc[6036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:59.598294 6142 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:07:59.598608 6142 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:59.598681 6142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:59.598834 6142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:59.599933 6142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "278967b96bb74c82bcc4d4b90db0fff7"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "e6a90f754ede9dc65d311bbdd5ed7999"
server_key_iv: "121dc5f39452d6548b2d16777087d1de"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.600298 6142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "278967b96bb74c82bcc4d4b90db0fff7"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "e6a90f754ede9dc65d311bbdd5ed7999"
server_key_iv: "121dc5f39452d6548b2d16777087d1de"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.601550 6142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:59.602228 6156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.602401 6142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:59.602506 6142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "278967b96bb74c82bcc4d4b90db0fff7"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "e6a90f754ede9dc65d311bbdd5ed7999"
server_key_iv: "121dc5f39452d6548b2d16777087d1de"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.602578 6142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:59.622087 6142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:59.624735 6142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:59.624897 6142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:59.625137 6142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:59.625455 6142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:59.625511 6142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.625554 6142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:59.625582 6142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.631178 6142 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:32771
I20260923 22:07:59.631212 6269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:32771 every 8 connection(s)
I20260923 22:07:59.631485 6142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:07:59 dist-test-slave-b5jz krb5kdc[6036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:59.640182 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.632263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:57321 (local address 127.1.244.254:42811)
0923 22:07:59.632331 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.632334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.632542 (+ 208us) server_negotiation.cc:407] Connection header received
0923 22:07:59.632993 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.632994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.633004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.633056 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.633907 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.634393 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.635323 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.635530 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.635633 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.637630 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.637631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.637631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.637638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.639030 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.639363 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.639363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.639364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.639413 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.639682 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.639682 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.639683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.639786 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.639838 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.640061 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.640121 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:07:59.640363 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.632248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42811 (local address 127.1.244.193:57321)
0923 22:07:59.632426 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.632460 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.632914 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.633125 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.633128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.633372 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:59.633833 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.633839 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.634470 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.634471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.635253 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.635255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.635644 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.635645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.635741 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.636171 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:59.636178 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:59.637553 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:59.639100 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.639105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.639106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.639300 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.639470 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.639471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.639471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.639616 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.639844 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:59.639848 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:59.639953 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.640033 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.640144 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260923 22:07:59.640851 6270 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42811
I20260923 22:07:59.640951 6270 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:59.641013 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 6142
I20260923 22:07:59.641114 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:59.641115 6270 heartbeater.cc:511] Master 127.1.244.254:42811 requested a full tablet report, sending...
I20260923 22:07:59.641343 2003 external_mini_cluster.cc:1428] Setting key cc83255f64f4b7ec771b3197ffc753b3
I20260923 22:07:59.641649 6080 ts_manager.cc:194] Registered new tserver with Master: 278967b96bb74c82bcc4d4b90db0fff7 (127.1.244.193:32771)
I20260923 22:07:59.642453 6080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:57321
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:07:59.664695 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42811
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:59.743310 6275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:59.743533 6275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:59.743583 6275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:59.744751 6275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:59.744819 6275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:59.744866 6275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:07:59.746395 6275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:59.746654 6275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:59.746870 6275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:59.751721 6283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.751727 6281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.751797 6275 server_base.cc:1034] running on GCE node
W20260923 22:07:59.751780 6280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.752099 6275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:59.752359 6275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:59.753522 6275 hybrid_clock.cc:648] HybridClock initialized: now 1790201279753518 us; error 36 us; skew 500 ppm
Sep 23 22:07:59 dist-test-slave-b5jz krb5kdc[6036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:59.755998 6275 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:07:59.756347 6275 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:59.756419 6275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:59.756575 6275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:59.757807 6275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "ed0d99e3fe904735a7db2c7bef4ac441"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "eec16f399a0d7cafc8c1a5be7aa9ac14"
server_key_iv: "f27b52146e4f7ae3458752a09ca7e5c7"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.758188 6275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "ed0d99e3fe904735a7db2c7bef4ac441"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "eec16f399a0d7cafc8c1a5be7aa9ac14"
server_key_iv: "f27b52146e4f7ae3458752a09ca7e5c7"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.759488 6275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:07:59.760213 6289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.760381 6275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:59.760473 6275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "ed0d99e3fe904735a7db2c7bef4ac441"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "eec16f399a0d7cafc8c1a5be7aa9ac14"
server_key_iv: "f27b52146e4f7ae3458752a09ca7e5c7"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.760553 6275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:59.771214 6275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:59.773025 6275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:59.773190 6275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:59.773417 6275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:59.773718 6275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:59.773772 6275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.773816 6275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:59.773844 6275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.779531 6275 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:33507
I20260923 22:07:59.779616 6402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:33507 every 8 connection(s)
I20260923 22:07:59.779908 6275 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:07:59 dist-test-slave-b5jz krb5kdc[6036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:59.788146 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.780622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:33325 (local address 127.1.244.254:42811)
0923 22:07:59.780666 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.780668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.780869 (+ 201us) server_negotiation.cc:407] Connection header received
0923 22:07:59.781334 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.781335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.781345 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.781396 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.782221 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.782706 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.783691 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.783876 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.783925 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.785821 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.785823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.785823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.785830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.786973 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.787328 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.787328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.787329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.787379 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.787639 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.787640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.787640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.787741 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.787796 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.788020 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.788084 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:07:59.788303 6405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.780587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42811 (local address 127.1.244.194:33325)
0923 22:07:59.780736 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.780785 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.781242 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.781453 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.781456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.781700 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:59.782150 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.782155 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.782789 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.782789 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.783609 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.783612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.783940 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.783940 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.784033 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.784435 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:59.784443 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:59.785735 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:59.787045 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.787050 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.787052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.787255 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.787432 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.787432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.787433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.787525 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.787801 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:59.787806 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:59.787913 (+ 107us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.787994 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.788106 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260923 22:07:59.788625 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 6275
I20260923 22:07:59.788743 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:59.788789 6403 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42811
I20260923 22:07:59.788880 6403 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:59.789000 2003 external_mini_cluster.cc:1428] Setting key c4eb4513b0275685e2eb8f945083863e
I20260923 22:07:59.789047 6403 heartbeater.cc:511] Master 127.1.244.254:42811 requested a full tablet report, sending...
I20260923 22:07:59.789455 6079 ts_manager.cc:194] Registered new tserver with Master: ed0d99e3fe904735a7db2c7bef4ac441 (127.1.244.194:33507)
I20260923 22:07:59.789983 6079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:33325
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:07:59.812776 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42811
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:59.892385 6408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:59.892584 6408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:59.892607 6408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:59.893895 6408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:59.893939 6408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:59.893965 6408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:07:59.895394 6408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:07:59.895639 6408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:59.895833 6408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:07:59.900918 6408 server_base.cc:1034] running on GCE node
W20260923 22:07:59.900866 6414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.900887 6413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.900866 6416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.901391 6408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:59.901600 6408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:59.902738 6408 hybrid_clock.cc:648] HybridClock initialized: now 1790201279902717 us; error 38 us; skew 500 ppm
Sep 23 22:07:59 dist-test-slave-b5jz krb5kdc[6036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:59.905165 6408 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:07:59.905577 6408 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:59.905627 6408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:59.905776 6408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:59.906836 6408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "ff857aa4c5b84a0c86d3545565aea23b"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "dba291353c51a31965ea0c7f12908921"
server_key_iv: "82b1ba20722f52524660e649ff0d4a18"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.907219 6408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "ff857aa4c5b84a0c86d3545565aea23b"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "dba291353c51a31965ea0c7f12908921"
server_key_iv: "82b1ba20722f52524660e649ff0d4a18"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.908593 6408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:07:59.909297 6422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.909474 6408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.909544 6408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "ff857aa4c5b84a0c86d3545565aea23b"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-b5jz"
server_key: "dba291353c51a31965ea0c7f12908921"
server_key_iv: "82b1ba20722f52524660e649ff0d4a18"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.909636 6408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:59.923321 6408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:59.925089 6408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:59.925248 6408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:59.925475 6408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:59.925787 6408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:59.925841 6408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.925881 6408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:59.925910 6408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.931429 6408 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:39435
I20260923 22:07:59.931466 6535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:39435 every 8 connection(s)
I20260923 22:07:59.931758 6408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:07:59.937320 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 6408
I20260923 22:07:59.937402 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
Sep 23 22:07:59 dist-test-slave-b5jz krb5kdc[6036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:07:59.937597 2003 external_mini_cluster.cc:1428] Setting key f188bb1f167b89334fc0265538baa30b
I20260923 22:07:59.940630 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.932449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:58891 (local address 127.1.244.254:42811)
0923 22:07:59.932530 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.932532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.932688 (+ 156us) server_negotiation.cc:407] Connection header received
0923 22:07:59.933132 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.933134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.933162 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.933229 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.934092 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.934572 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.935493 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.935832 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.935886 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.938193 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.938195 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.938195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.938203 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.939488 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.939885 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.939885 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.939886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.939935 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.940138 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.940138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.940138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.940234 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.940286 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.940510 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.940569 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:07:59.940779 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.932404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42811 (local address 127.1.244.195:58891)
0923 22:07:59.932573 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.932607 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.933056 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.933297 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.933300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.933547 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:59.934014 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.934019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.934647 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.934648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.935420 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.935422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.935908 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.935909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.936030 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.936494 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:59.936502 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:59.938106 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:59.939599 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.939604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.939606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.939821 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.939992 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.939993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.939993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.940085 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.940295 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:59.940298 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:59.940404 (+ 106us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.940482 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.940593 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260923 22:07:59.941191 6536 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42811
I20260923 22:07:59.941267 6536 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:59.941373 6536 heartbeater.cc:511] Master 127.1.244.254:42811 requested a full tablet report, sending...
I20260923 22:07:59.941653 6079 ts_manager.cc:194] Registered new tserver with Master: ff857aa4c5b84a0c86d3545565aea23b (127.1.244.195:39435)
I20260923 22:07:59.942025 6079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:58891
I20260923 22:07:59.949316 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:07:59.956032 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.950558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35008 (local address 127.1.244.254:42811)
0923 22:07:59.950615 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.950617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.950721 (+ 104us) server_negotiation.cc:407] Connection header received
0923 22:07:59.950766 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.950766 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.950775 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.950823 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.951359 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.951890 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.952566 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.952735 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.952790 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.953913 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.953914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.953914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.953919 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.955025 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.955322 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.955323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.955323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.955370 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.955526 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.955526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.955527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.955724 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.955810 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.955913 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.955970 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:07:59.957705 6079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:07:59.959882 6079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:59.966308 6548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.962171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.1.244.195:39435)
0923 22:07:59.962343 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.962348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.962840 (+ 492us) server_negotiation.cc:407] Connection header received
0923 22:07:59.962879 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.962881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.962895 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.962988 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:59.963127 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.963928 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.965237 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.965951 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.966030 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.966087 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.966172 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.966212 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260923 22:07:59.966311 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.962100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39435 (local address 127.0.0.1:58916)
0923 22:07:59.962707 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.962717 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.962767 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.963007 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.963008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.963010 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:59.963056 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.963057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.964029 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.964030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.965149 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.965151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.966035 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.966036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.966091 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.966102 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.966157 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.966228 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":76,"thread_start_us":51,"threads_started":1}
I20260923 22:07:59.966442 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.961906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32771 (local address 127.0.0.1:59062)
0923 22:07:59.962299 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.962309 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.962350 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.962594 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.962598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.962600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:59.962660 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.962662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.963985 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.963986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.965346 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.965349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.966211 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.966212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.966273 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.966277 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.966337 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.966376 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":60,"thread_start_us":52,"threads_started":1}
I20260923 22:07:59.966677 6546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.962010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.1.244.193:32771)
0923 22:07:59.962366 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.962371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.962394 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:07:59.962432 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.962434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.962448 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.962543 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:59.962753 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.963887 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.965432 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.966133 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.966199 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.966246 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.966547 (+ 301us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.966594 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":74,"threads_started":1}
I20260923 22:07:59.967307 6204 tablet_service.cc:1467] Processing CreateTablet for tablet 199a81688f994b36916348049f259133 (DEFAULT_TABLE table=test-table [id=d902d144c2ed47e3aee82e469efdcd6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:59.967648 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.961905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:33507 (local address 127.0.0.1:51862)
0923 22:07:59.962067 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.962109 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.962176 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.962759 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.962760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.962762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:59.962803 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.962804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.964101 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.964101 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.965347 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.965349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.967381 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.967382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.967444 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.967447 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.967511 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.967575 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":86,"thread_start_us":59,"threads_started":1}
I20260923 22:07:59.967721 6204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199a81688f994b36916348049f259133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.967844 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.962235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.1.244.194:33507)
0923 22:07:59.962435 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.962438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.962446 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:07:59.962477 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.962479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.962491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.962603 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:59.962882 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.963987 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.966282 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.967292 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.967365 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.967421 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.967587 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.967637 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260923 22:07:59.967780 6470 tablet_service.cc:1467] Processing CreateTablet for tablet 199a81688f994b36916348049f259133 (DEFAULT_TABLE table=test-table [id=d902d144c2ed47e3aee82e469efdcd6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:59.968087 6470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199a81688f994b36916348049f259133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.970409 6550 tablet_bootstrap.cc:492] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7: Bootstrap starting.
I20260923 22:07:59.970695 6551 tablet_bootstrap.cc:492] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: Bootstrap starting.
I20260923 22:07:59.971102 6550 tablet_bootstrap.cc:654] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.971269 6551 tablet_bootstrap.cc:654] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.971517 6337 tablet_service.cc:1467] Processing CreateTablet for tablet 199a81688f994b36916348049f259133 (DEFAULT_TABLE table=test-table [id=d902d144c2ed47e3aee82e469efdcd6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:59.971853 6550 log.cc:824] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:59.971916 6337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199a81688f994b36916348049f259133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.971948 6551 log.cc:824] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:59.972601 6550 tablet_bootstrap.cc:492] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7: No bootstrap required, opened a new log
I20260923 22:07:59.972646 6550 ts_tablet_manager.cc:1397] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:59.973930 6550 raft_consensus.cc:348] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } }
I20260923 22:07:59.974049 6550 raft_consensus.cc:374] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.974069 6550 raft_consensus.cc:729] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278967b96bb74c82bcc4d4b90db0fff7, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.974188 6550 consensus_queue.cc:260] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } }
I20260923 22:07:59.974295 6554 tablet_bootstrap.cc:492] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: Bootstrap starting.
I20260923 22:07:59.974470 6550 ts_tablet_manager.cc:1428] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:59.974638 6270 heartbeater.cc:503] Master 127.1.244.254:42811 was elected leader, sending a full tablet report...
I20260923 22:07:59.974933 6554 tablet_bootstrap.cc:654] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.975147 6551 tablet_bootstrap.cc:492] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: No bootstrap required, opened a new log
I20260923 22:07:59.975220 6551 ts_tablet_manager.cc:1397] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260923 22:07:59.975610 6554 log.cc:824] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:59.976305 6554 tablet_bootstrap.cc:492] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: No bootstrap required, opened a new log
I20260923 22:07:59.976357 6554 ts_tablet_manager.cc:1397] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:59.976948 6551 raft_consensus.cc:348] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } }
I20260923 22:07:59.977093 6551 raft_consensus.cc:374] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.977139 6551 raft_consensus.cc:729] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff857aa4c5b84a0c86d3545565aea23b, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.977263 6551 consensus_queue.cc:260] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } }
I20260923 22:07:59.977553 6551 ts_tablet_manager.cc:1428] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:59.977587 6536 heartbeater.cc:503] Master 127.1.244.254:42811 was elected leader, sending a full tablet report...
I20260923 22:07:59.978044 6554 raft_consensus.cc:348] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } }
I20260923 22:07:59.978220 6554 raft_consensus.cc:374] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.978256 6554 raft_consensus.cc:729] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0d99e3fe904735a7db2c7bef4ac441, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.978367 6554 consensus_queue.cc:260] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } }
I20260923 22:07:59.978641 6554 ts_tablet_manager.cc:1428] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:59.978641 6403 heartbeater.cc:503] Master 127.1.244.254:42811 was elected leader, sending a full tablet report...
W20260923 22:08:00.030437 6404 tablet.cc:2369] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:08:00.132336 6271 tablet.cc:2369] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:00.179605 6555 raft_consensus.cc:482] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:00.179734 6555 raft_consensus.cc:504] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } }
I20260923 22:08:00.180035 6555 leader_election.cc:288] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed0d99e3fe904735a7db2c7bef4ac441 (127.1.244.194:33507), ff857aa4c5b84a0c86d3545565aea23b (127.1.244.195:39435)
W20260923 22:08:00.182238 6537 tablet.cc:2369] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:00.183421 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.180345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:47283 (local address 127.1.244.194:33507)
0923 22:08:00.180422 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.180424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.180516 (+ 92us) server_negotiation.cc:407] Connection header received
0923 22:08:00.180651 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.180652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.180669 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.180747 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:00.180905 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.181369 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.182461 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.183055 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.183128 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.183136 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.183294 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.183336 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 22:08:00.183421 6548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.180548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:49203 (local address 127.1.244.195:39435)
0923 22:08:00.180617 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.180620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.180629 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:00.180658 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.180658 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.180667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.180724 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:00.180877 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.181357 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.182471 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.183050 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.183128 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.183136 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.183303 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.183334 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:08:00.183451 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.180242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:33507 (local address 127.1.244.193:47283)
0923 22:08:00.180428 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.180440 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.180548 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.180755 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.180755 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.180757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:00.180829 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.180836 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.181455 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.181456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.182371 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.182374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.183140 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.183142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.183218 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.183221 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.183274 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.183326 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":47,"thread_start_us":78,"threads_started":1}
I20260923 22:08:00.183451 6560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.180454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39435 (local address 127.1.244.193:49203)
0923 22:08:00.180533 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.180543 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.180603 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.180728 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.180729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.180731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:00.180800 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.180802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.181441 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.181442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.182270 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.182272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.183142 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.183143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.183207 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.183212 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.183274 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.183326 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":24,"thread_start_us":43,"threads_started":1}
I20260923 22:08:00.183859 6357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199a81688f994b36916348049f259133" candidate_uuid: "278967b96bb74c82bcc4d4b90db0fff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" is_pre_election: true
I20260923 22:08:00.183928 6490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199a81688f994b36916348049f259133" candidate_uuid: "278967b96bb74c82bcc4d4b90db0fff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff857aa4c5b84a0c86d3545565aea23b" is_pre_election: true
I20260923 22:08:00.184034 6357 raft_consensus.cc:2454] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 278967b96bb74c82bcc4d4b90db0fff7 in term 0.
I20260923 22:08:00.184053 6490 raft_consensus.cc:2454] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 278967b96bb74c82bcc4d4b90db0fff7 in term 0.
I20260923 22:08:00.184257 6160 leader_election.cc:302] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278967b96bb74c82bcc4d4b90db0fff7, ff857aa4c5b84a0c86d3545565aea23b; no voters:
I20260923 22:08:00.184417 6555 raft_consensus.cc:2781] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:00.184489 6555 raft_consensus.cc:482] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:00.184527 6555 raft_consensus.cc:3037] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.185189 6555 raft_consensus.cc:504] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } }
I20260923 22:08:00.185322 6555 leader_election.cc:288] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [CANDIDATE]: Term 1 election: Requested vote from peers ed0d99e3fe904735a7db2c7bef4ac441 (127.1.244.194:33507), ff857aa4c5b84a0c86d3545565aea23b (127.1.244.195:39435)
I20260923 22:08:00.185489 6357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199a81688f994b36916348049f259133" candidate_uuid: "278967b96bb74c82bcc4d4b90db0fff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0d99e3fe904735a7db2c7bef4ac441"
I20260923 22:08:00.185503 6490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199a81688f994b36916348049f259133" candidate_uuid: "278967b96bb74c82bcc4d4b90db0fff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff857aa4c5b84a0c86d3545565aea23b"
I20260923 22:08:00.185595 6490 raft_consensus.cc:3037] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.185601 6357 raft_consensus.cc:3037] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.186269 6357 raft_consensus.cc:2454] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278967b96bb74c82bcc4d4b90db0fff7 in term 1.
I20260923 22:08:00.186300 6490 raft_consensus.cc:2454] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 278967b96bb74c82bcc4d4b90db0fff7 in term 1.
I20260923 22:08:00.186450 6160 leader_election.cc:302] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278967b96bb74c82bcc4d4b90db0fff7, ff857aa4c5b84a0c86d3545565aea23b; no voters:
I20260923 22:08:00.186555 6555 raft_consensus.cc:2781] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:00.186709 6555 raft_consensus.cc:686] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 1 LEADER]: Becoming Leader. State: Replica: 278967b96bb74c82bcc4d4b90db0fff7, State: Running, Role: LEADER
I20260923 22:08:00.186818 6555 consensus_queue.cc:237] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } }
I20260923 22:08:00.187695 6080 catalog_manager.cc:5495] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 278967b96bb74c82bcc4d4b90db0fff7 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "278967b96bb74c82bcc4d4b90db0fff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278967b96bb74c82bcc4d4b90db0fff7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:00.248656 6546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.246192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.1.244.193:32771)
0923 22:08:00.246265 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.246268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.246278 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:00.246344 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.246345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.246369 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.246443 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:00.246606 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.247085 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.247874 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.248066 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.248140 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.248309 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:00.248421 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:00.248518 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.248541 (+ 23us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.248563 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:08:00.250620 6357 raft_consensus.cc:1261] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [term 1 FOLLOWER]: Refusing update from remote peer 278967b96bb74c82bcc4d4b90db0fff7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:00.250659 6490 raft_consensus.cc:1261] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [term 1 FOLLOWER]: Refusing update from remote peer 278967b96bb74c82bcc4d4b90db0fff7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:00.250989 6561 consensus_queue.cc:1035] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0d99e3fe904735a7db2c7bef4ac441" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:00.251116 6555 consensus_queue.cc:1035] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff857aa4c5b84a0c86d3545565aea23b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:00.253567 6564 mvcc.cc:205] Tried to move back new op lower bound from 7332664443903873024 to 7332664443645857792. Current Snapshot: MvccSnapshot[applied={T|T < 7332664443903873024}]
I20260923 22:08:00.256481 6080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:00.256573 6080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:00.257310 6080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6d05884572d4d25a20583db1a42bef1: Sending DeleteTablet for 3 replicas of tablet 199a81688f994b36916348049f259133
I20260923 22:08:00.257613 6204 tablet_service.cc:1514] Processing DeleteTablet for tablet 199a81688f994b36916348049f259133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:00 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:59062
I20260923 22:08:00.257660 6337 tablet_service.cc:1514] Processing DeleteTablet for tablet 199a81688f994b36916348049f259133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:00 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:51862
I20260923 22:08:00.257880 6574 tablet_replica.cc:330] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: stopping tablet replica
I20260923 22:08:00.257898 6573 tablet_replica.cc:330] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7: stopping tablet replica
I20260923 22:08:00.257879 6470 tablet_service.cc:1514] Processing DeleteTablet for tablet 199a81688f994b36916348049f259133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:00 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:58916
I20260923 22:08:00.257979 6573 raft_consensus.cc:2229] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:00.257980 6574 raft_consensus.cc:2229] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:00.258070 6574 raft_consensus.cc:2258] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:00.258077 6573 raft_consensus.cc:2258] T 199a81688f994b36916348049f259133 P 278967b96bb74c82bcc4d4b90db0fff7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:00.258419 6575 tablet_replica.cc:330] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: stopping tablet replica
I20260923 22:08:00.258512 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 6142
I20260923 22:08:00.258554 6575 raft_consensus.cc:2229] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:00.258664 6575 raft_consensus.cc:2258] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:00.259289 6575 ts_tablet_manager.cc:1905] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:00.260515 6574 ts_tablet_manager.cc:1905] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:00.261155 6575 ts_tablet_manager.cc:1918] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:00.261236 6575 log.cc:1196] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/199a81688f994b36916348049f259133
I20260923 22:08:00.261575 6575 ts_tablet_manager.cc:1939] T 199a81688f994b36916348049f259133 P ff857aa4c5b84a0c86d3545565aea23b: Deleting consensus metadata
I20260923 22:08:00.261998 6067 catalog_manager.cc:4841] TS ff857aa4c5b84a0c86d3545565aea23b (127.1.244.195:39435): tablet 199a81688f994b36916348049f259133 (table test-table [id=d902d144c2ed47e3aee82e469efdcd6e]) successfully deleted
I20260923 22:08:00.262158 6574 ts_tablet_manager.cc:1918] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:00.262251 6574 log.cc:1196] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/199a81688f994b36916348049f259133
I20260923 22:08:00.262473 6574 ts_tablet_manager.cc:1939] T 199a81688f994b36916348049f259133 P ed0d99e3fe904735a7db2c7bef4ac441: Deleting consensus metadata
I20260923 22:08:00.262859 6067 catalog_manager.cc:4841] TS ed0d99e3fe904735a7db2c7bef4ac441 (127.1.244.194:33507): tablet 199a81688f994b36916348049f259133 (table test-table [id=d902d144c2ed47e3aee82e469efdcd6e]) successfully deleted
W20260923 22:08:00.264609 6064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:32771 (error 108)
I20260923 22:08:00.264834 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 6275
I20260923 22:08:00.265412 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.264967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32771 (local address 127.0.0.1:59084)
0923 22:08:00.265334 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.265372 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347}
W20260923 22:08:00.265703 6064 catalog_manager.cc:4584] TS 278967b96bb74c82bcc4d4b90db0fff7 (127.1.244.193:32771): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 199a81688f994b36916348049f259133: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32771: connect: Connection refused (error 111)
I20260923 22:08:00.269415 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 6408
I20260923 22:08:00.273720 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 6049
2026-09-23T22:08:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2082 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-23T22:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:00Z Disabled control of system clock
I20260923 22:08:00.300453 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38183
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:34949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:08:00.378903 6580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:00.379088 6580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:00.379108 6580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:00.380199 6580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:00.380234 6580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:00.380246 6580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:00.380259 6580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:00.380271 6580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:00.381529 6580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38183
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:00.381706 6580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:00.381923 6580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:00.386520 6586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:00.386507 6588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:00.386723 6580 server_base.cc:1034] running on GCE node
W20260923 22:08:00.386499 6585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:00.387025 6580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:00.387347 6580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:00.388507 6580 hybrid_clock.cc:648] HybridClock initialized: now 1790201280388500 us; error 36 us; skew 500 ppm
I20260923 22:08:00.389657 6580 webserver.cc:460] Webserver started at http://127.1.244.254:34277/ using document root <none> and password file <none>
I20260923 22:08:00.389911 6580 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:00.389982 6580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:00.390123 6580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:00.391182 6580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "daf5286a9e084cec95f0fd86855de045"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "11ea0953b746492195adb28c4ed37ad5"
server_key_iv: "559b1eac8de0330ee9ac88bf3b4a8d57"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.391593 6580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "daf5286a9e084cec95f0fd86855de045"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "11ea0953b746492195adb28c4ed37ad5"
server_key_iv: "559b1eac8de0330ee9ac88bf3b4a8d57"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.392874 6580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:00.393549 6594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.393725 6580 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:00.393826 6580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "daf5286a9e084cec95f0fd86855de045"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "11ea0953b746492195adb28c4ed37ad5"
server_key_iv: "559b1eac8de0330ee9ac88bf3b4a8d57"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.393908 6580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:00.397094 6580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:00.397354 6580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:00.397497 6580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:00.401368 6580 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:38183
I20260923 22:08:00.401400 6646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:38183 every 8 connection(s)
I20260923 22:08:00.401708 6580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:00.402359 6647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:00.404242 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 6580
I20260923 22:08:00.404345 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:00.404553 2003 external_mini_cluster.cc:1428] Setting key 3bc023799d6c630bbf8798a664f950ff
I20260923 22:08:00.405462 6647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045: Bootstrap starting.
I20260923 22:08:00.406121 6647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:00.406471 6647 log.cc:824] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:00.407203 6647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045: No bootstrap required, opened a new log
I20260923 22:08:00.407426 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.405077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.1.244.254:38183)
0923 22:08:00.405369 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.405377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.405407 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:08:00.405978 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.405986 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.406038 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.406455 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.406921 (+ 466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.406924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.406925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:00.406930 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.406977 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:00.406987 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.407080 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.407134 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.407191 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
I20260923 22:08:00.409220 6647 raft_consensus.cc:348] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf5286a9e084cec95f0fd86855de045" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38183 } }
I20260923 22:08:00.409411 6647 raft_consensus.cc:374] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:00.409437 6647 raft_consensus.cc:729] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daf5286a9e084cec95f0fd86855de045, State: Initialized, Role: FOLLOWER
I20260923 22:08:00.409574 6647 consensus_queue.cc:260] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf5286a9e084cec95f0fd86855de045" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38183 } }
I20260923 22:08:00.409652 6647 raft_consensus.cc:388] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:00.409698 6647 raft_consensus.cc:482] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:00.409746 6647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.410245 6647 raft_consensus.cc:504] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf5286a9e084cec95f0fd86855de045" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38183 } }
I20260923 22:08:00.410363 6647 leader_election.cc:302] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daf5286a9e084cec95f0fd86855de045; no voters:
I20260923 22:08:00.410524 6647 leader_election.cc:288] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:00.410575 6652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:00.410753 6647 sys_catalog.cc:564] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:00.410759 6652 raft_consensus.cc:686] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [term 1 LEADER]: Becoming Leader. State: Replica: daf5286a9e084cec95f0fd86855de045, State: Running, Role: LEADER
I20260923 22:08:00.410874 6652 consensus_queue.cc:237] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf5286a9e084cec95f0fd86855de045" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38183 } }
I20260923 22:08:00.411247 6654 sys_catalog.cc:455] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [sys.catalog]: SysCatalogTable state changed. Reason: New leader daf5286a9e084cec95f0fd86855de045. Latest consensus state: current_term: 1 leader_uuid: "daf5286a9e084cec95f0fd86855de045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf5286a9e084cec95f0fd86855de045" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38183 } } }
I20260923 22:08:00.411358 6654 sys_catalog.cc:458] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:00.411516 6653 sys_catalog.cc:455] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daf5286a9e084cec95f0fd86855de045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf5286a9e084cec95f0fd86855de045" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38183 } } }
I20260923 22:08:00.411610 6653 sys_catalog.cc:458] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:00.411671 6661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:00.412147 6661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:00.413568 6661 catalog_manager.cc:1347] Generated new cluster ID: 0fb5e59c225546b7a6345c7790d7f769
I20260923 22:08:00.413636 6661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:00.430332 6661 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:00.430493 6661 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:00.436156 6661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045: Generated new TSK 0
I20260923 22:08:00.436352 6661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:00.437808 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38183
--builtin_ntp_servers=127.1.244.212:34949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:08:00.518436 6671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:00.518659 6671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:00.518694 6671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:00.519873 6671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:00.519930 6671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:00.519971 6671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:00.521286 6671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:00.521543 6671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:00.521831 6671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:00.527519 6677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:00.527704 6671 server_base.cc:1034] running on GCE node
W20260923 22:08:00.527526 6676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:00.527686 6679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:00.528036 6671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:00.528306 6671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:00.529459 6671 hybrid_clock.cc:648] HybridClock initialized: now 1790201280529449 us; error 27 us; skew 500 ppm
I20260923 22:08:00.530741 6671 webserver.cc:460] Webserver started at http://127.1.244.193:35859/ using document root <none> and password file <none>
I20260923 22:08:00.530999 6671 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:00.531042 6671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:00.531177 6671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:00.532284 6671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "66779f1f640042ab804ed2e6ee641d23"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "babca38a1cd1391ff7a5c85a0cfeed19"
server_key_iv: "228c2dfd0a0d8b75cd184cf6a48b2f5d"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.532624 6671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "66779f1f640042ab804ed2e6ee641d23"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "babca38a1cd1391ff7a5c85a0cfeed19"
server_key_iv: "228c2dfd0a0d8b75cd184cf6a48b2f5d"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.533929 6671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:00.534713 6685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.534898 6671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:00.534993 6671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "66779f1f640042ab804ed2e6ee641d23"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "babca38a1cd1391ff7a5c85a0cfeed19"
server_key_iv: "228c2dfd0a0d8b75cd184cf6a48b2f5d"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.535079 6671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:00.538371 6671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:00.538623 6671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:00.538772 6671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:00.538972 6671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:00.539263 6671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:00.539318 6671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.539356 6671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:00.539384 6671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.545356 6671 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:42843
I20260923 22:08:00.545387 6798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:42843 every 8 connection(s)
I20260923 22:08:00.545686 6671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:00.548372 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.546388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:45013 (local address 127.1.244.254:38183)
0923 22:08:00.546426 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.546428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.546653 (+ 225us) server_negotiation.cc:407] Connection header received
0923 22:08:00.547041 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.547042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.547047 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.547097 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.548180 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.548181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.548182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:00.548185 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.548200 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:00.548203 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.548246 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.548309 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.548321 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:00.548528 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.546394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38183 (local address 127.1.244.193:45013)
0923 22:08:00.546539 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.546568 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.546966 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.547143 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.547147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.547155 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:00.548054 (+ 899us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:00.548054 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:00.548080 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:00.548104 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:00.548251 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:00.548253 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:00.548255 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.548303 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.548346 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":78,"threads_started":1}
I20260923 22:08:00.549017 6799 heartbeater.cc:347] Connected to a master server at 127.1.244.254:38183
I20260923 22:08:00.549088 6799 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:00.549242 6799 heartbeater.cc:511] Master 127.1.244.254:38183 requested a full tablet report, sending...
I20260923 22:08:00.549736 6611 ts_manager.cc:194] Registered new tserver with Master: 66779f1f640042ab804ed2e6ee641d23 (127.1.244.193:42843)
I20260923 22:08:00.552488 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 6671
I20260923 22:08:00.552592 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:00.552785 2003 external_mini_cluster.cc:1428] Setting key 909689a036fb1335dd8fe27026d4c733
I20260923 22:08:00.553622 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38183
--builtin_ntp_servers=127.1.244.212:34949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:08:00.632464 6802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:00.632656 6802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:00.632678 6802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:00.633742 6802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:00.633783 6802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:00.633808 6802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:00.635046 6802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:00.635215 6802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:00.635432 6802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:00.640172 6807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:00.640178 6810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:00.640221 6802 server_base.cc:1034] running on GCE node
W20260923 22:08:00.640205 6808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:00.640604 6802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:00.640846 6802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:00.642006 6802 hybrid_clock.cc:648] HybridClock initialized: now 1790201280641990 us; error 28 us; skew 500 ppm
I20260923 22:08:00.643193 6802 webserver.cc:460] Webserver started at http://127.1.244.194:33463/ using document root <none> and password file <none>
I20260923 22:08:00.643451 6802 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:00.643522 6802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:00.643714 6802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:00.644728 6802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "45973c9ee3bd45f599c65f7c205e6ab4"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "b7398a9461ab10a8efb35dcc23e1c9fc"
server_key_iv: "b44ed208b5e837df7eb44120203936e2"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.645084 6802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "45973c9ee3bd45f599c65f7c205e6ab4"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "b7398a9461ab10a8efb35dcc23e1c9fc"
server_key_iv: "b44ed208b5e837df7eb44120203936e2"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.646337 6802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:00.647001 6816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.647173 6802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:00.647274 6802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "45973c9ee3bd45f599c65f7c205e6ab4"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "b7398a9461ab10a8efb35dcc23e1c9fc"
server_key_iv: "b44ed208b5e837df7eb44120203936e2"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.647354 6802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:00.650548 6802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:00.650799 6802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:00.650936 6802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:00.651141 6802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:00.651436 6802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:00.651489 6802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.651527 6802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:00.651575 6802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.657138 6802 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:33687
I20260923 22:08:00.657179 6929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:33687 every 8 connection(s)
I20260923 22:08:00.657469 6802 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:00.657806 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 6802
I20260923 22:08:00.657907 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:00.658114 2003 external_mini_cluster.cc:1428] Setting key 9d13a0be4b813a82c59977e609cbe3d6
I20260923 22:08:00.659085 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38183
--builtin_ntp_servers=127.1.244.212:34949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 22:08:00.660532 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.658144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:45951 (local address 127.1.244.254:38183)
0923 22:08:00.658320 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.658322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.658460 (+ 138us) server_negotiation.cc:407] Connection header received
0923 22:08:00.658936 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.658937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.658944 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.658990 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.660271 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.660273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.660273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:00.660279 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.660300 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:00.660305 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.660371 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.660466 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.660479 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260923 22:08:00.660750 6932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.658129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38183 (local address 127.1.244.194:45951)
0923 22:08:00.658335 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.658376 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.658847 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.659050 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.659054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.659064 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:00.660064 (+ 1000us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:00.660064 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:00.660090 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:00.660149 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:00.660380 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:00.660386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:00.660389 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.660468 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.660531 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260923 22:08:00.661263 6930 heartbeater.cc:347] Connected to a master server at 127.1.244.254:38183
I20260923 22:08:00.661360 6930 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:00.661458 6930 heartbeater.cc:511] Master 127.1.244.254:38183 requested a full tablet report, sending...
I20260923 22:08:00.661749 6611 ts_manager.cc:194] Registered new tserver with Master: 45973c9ee3bd45f599c65f7c205e6ab4 (127.1.244.194:33687)
W20260923 22:08:00.738552 6933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:00.738782 6933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:00.738835 6933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:00.740038 6933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:00.740075 6933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:00.740128 6933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:00.741420 6933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:00.741662 6933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:00.741943 6933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:00.746794 6939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:00.746824 6941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:00.746824 6938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:00.746928 6933 server_base.cc:1034] running on GCE node
I20260923 22:08:00.747124 6933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:00.747351 6933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:00.748517 6933 hybrid_clock.cc:648] HybridClock initialized: now 1790201280748496 us; error 34 us; skew 500 ppm
I20260923 22:08:00.749907 6933 webserver.cc:460] Webserver started at http://127.1.244.195:46075/ using document root <none> and password file <none>
I20260923 22:08:00.750164 6933 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:00.750237 6933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:00.750382 6933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:00.751443 6933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "62dfbee4bcb34ff8bdf49b034594544b"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "4a3d6de3938591208d401727745b88bd"
server_key_iv: "ca0c505918883316931b9b014b410db2"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.751824 6933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "62dfbee4bcb34ff8bdf49b034594544b"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "4a3d6de3938591208d401727745b88bd"
server_key_iv: "ca0c505918883316931b9b014b410db2"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.753077 6933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:00.753788 6947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.753947 6933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:00.754027 6933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "62dfbee4bcb34ff8bdf49b034594544b"
format_stamp: "Formatted at 2026-09-23 22:08:00 on dist-test-slave-b5jz"
server_key: "4a3d6de3938591208d401727745b88bd"
server_key_iv: "ca0c505918883316931b9b014b410db2"
server_key_version: "encryptionkey@0"
I20260923 22:08:00.754079 6933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:00.757376 6933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:00.757584 6933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:00.757673 6933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:00.757817 6933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:00.758039 6933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:00.758066 6933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.758086 6933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:00.758098 6933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.763737 6933 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:43049
I20260923 22:08:00.763780 7060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:43049 every 8 connection(s)
I20260923 22:08:00.764060 6933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:00.767019 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.764740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:38341 (local address 127.1.244.254:38183)
0923 22:08:00.764828 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.764830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.765018 (+ 188us) server_negotiation.cc:407] Connection header received
0923 22:08:00.765504 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.765506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.765513 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.765571 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.766750 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.766752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.766752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:00.766764 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.766778 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:00.766783 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.766836 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.766934 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.766949 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:08:00.767234 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.764720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38183 (local address 127.1.244.195:38341)
0923 22:08:00.764905 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.764936 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.765389 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.765646 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.765650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.765658 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:00.766593 (+ 935us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:00.766594 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:00.766621 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:00.766646 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:00.766842 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:00.766846 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:00.766849 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.766944 (+ 95us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.767003 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260923 22:08:00.767764 7061 heartbeater.cc:347] Connected to a master server at 127.1.244.254:38183
I20260923 22:08:00.767836 7061 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:00.767993 7061 heartbeater.cc:511] Master 127.1.244.254:38183 requested a full tablet report, sending...
I20260923 22:08:00.768340 6611 ts_manager.cc:194] Registered new tserver with Master: 62dfbee4bcb34ff8bdf49b034594544b (127.1.244.195:43049)
I20260923 22:08:00.774085 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 6933
I20260923 22:08:00.774166 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:00.774358 2003 external_mini_cluster.cc:1428] Setting key 601747c9b9afbb0aa76a3d0d5e71a297
I20260923 22:08:00.775290 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:08:00.777652 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.776617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55848 (local address 127.1.244.254:38183)
0923 22:08:00.776701 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.776703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.776719 (+ 16us) server_negotiation.cc:407] Connection header received
0923 22:08:00.776782 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.776783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.776787 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.776837 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.777434 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.777435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.777435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:00.777438 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.777447 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:00.777450 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.777483 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.777538 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.777547 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 22:08:00.779024 6611 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:00.780647 6611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:00.784422 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.782713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:43049 (local address 127.0.0.1:41448)
0923 22:08:00.782880 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.782916 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.782969 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.783459 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.783465 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.783471 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:00.783952 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:00.783952 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:00.783968 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:00.783992 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:00.784239 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:00.784245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:00.784247 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.784318 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.784331 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260923 22:08:00.784446 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.782870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.1.244.195:43049)
0923 22:08:00.783030 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.783035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.783046 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:08:00.783070 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.783072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.783081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.783152 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.784079 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.784088 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.784089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:00.784093 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.784131 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:00.784155 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.784214 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.784344 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.784360 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
I20260923 22:08:00.784520 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.782909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:33687 (local address 127.0.0.1:54644)
0923 22:08:00.783057 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.783065 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.783097 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.783649 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.783650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.783653 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:00.784106 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:00.784106 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:00.784112 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:00.784122 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:00.784365 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:00.784366 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:00.784367 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.784396 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.784409 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":62,"threads_started":1}
I20260923 22:08:00.784732 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.783106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54644 (local address 127.1.244.194:33687)
0923 22:08:00.783282 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.783287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.783467 (+ 180us) server_negotiation.cc:407] Connection header received
0923 22:08:00.783492 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.783494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.783503 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.783810 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.784200 (+ 390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.784207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.784208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:00.784211 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.784242 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:00.784271 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.784604 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.784630 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.784649 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260923 22:08:00.785180 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.782714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42843 (local address 127.0.0.1:42148)
0923 22:08:00.783019 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.783030 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.783081 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.784131 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.784132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.784134 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:00.784776 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:00.784776 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:00.784782 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:00.784818 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:00.785058 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:00.785059 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:00.785060 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.785107 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.785121 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":4,"thread_start_us":69,"threads_started":1}
I20260923 22:08:00.785223 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.782809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.1.244.193:42843)
0923 22:08:00.783329 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.783332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.783343 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:08:00.783368 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.783370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.783378 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.783453 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.784909 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.784917 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.784917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:00.784922 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.784956 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:00.784981 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.785063 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.785123 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.785138 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":94,"threads_started":1}
I20260923 22:08:00.785246 6864 tablet_service.cc:1467] Processing CreateTablet for tablet 2a94a078d770492eba48eec4637e3242 (DEFAULT_TABLE table=test-table [id=a4600d522522425281bb59b35372a72d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:00.785243 6995 tablet_service.cc:1467] Processing CreateTablet for tablet 2a94a078d770492eba48eec4637e3242 (DEFAULT_TABLE table=test-table [id=a4600d522522425281bb59b35372a72d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:00.785575 6864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a94a078d770492eba48eec4637e3242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:00.785575 6995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a94a078d770492eba48eec4637e3242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:00.786031 6733 tablet_service.cc:1467] Processing CreateTablet for tablet 2a94a078d770492eba48eec4637e3242 (DEFAULT_TABLE table=test-table [id=a4600d522522425281bb59b35372a72d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:00.786331 6733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a94a078d770492eba48eec4637e3242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:00.788770 7075 tablet_bootstrap.cc:492] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23: Bootstrap starting.
I20260923 22:08:00.788987 7077 tablet_bootstrap.cc:492] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: Bootstrap starting.
I20260923 22:08:00.789482 7075 tablet_bootstrap.cc:654] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:00.789542 7077 tablet_bootstrap.cc:654] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:00.789783 7075 log.cc:824] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:00.789813 7077 log.cc:824] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:00.790133 7076 tablet_bootstrap.cc:492] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: Bootstrap starting.
I20260923 22:08:00.790488 7075 tablet_bootstrap.cc:492] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23: No bootstrap required, opened a new log
I20260923 22:08:00.790540 7075 ts_tablet_manager.cc:1397] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:00.790685 7076 tablet_bootstrap.cc:654] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:00.790954 7076 log.cc:824] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:00.791402 7077 tablet_bootstrap.cc:492] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: No bootstrap required, opened a new log
I20260923 22:08:00.791474 7077 ts_tablet_manager.cc:1397] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:08:00.791708 7076 tablet_bootstrap.cc:492] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: No bootstrap required, opened a new log
I20260923 22:08:00.791775 7076 ts_tablet_manager.cc:1397] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:00.792928 7075 raft_consensus.cc:348] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } }
I20260923 22:08:00.793075 7075 raft_consensus.cc:374] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:00.793118 7075 raft_consensus.cc:729] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66779f1f640042ab804ed2e6ee641d23, State: Initialized, Role: FOLLOWER
I20260923 22:08:00.793239 7075 consensus_queue.cc:260] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } }
I20260923 22:08:00.793498 7075 ts_tablet_manager.cc:1428] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:08:00.793648 6799 heartbeater.cc:503] Master 127.1.244.254:38183 was elected leader, sending a full tablet report...
I20260923 22:08:00.794122 7077 raft_consensus.cc:348] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } }
I20260923 22:08:00.794282 7077 raft_consensus.cc:374] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:00.794317 7077 raft_consensus.cc:729] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45973c9ee3bd45f599c65f7c205e6ab4, State: Initialized, Role: FOLLOWER
I20260923 22:08:00.794243 7076 raft_consensus.cc:348] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } }
I20260923 22:08:00.794382 7076 raft_consensus.cc:374] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:00.794430 7076 raft_consensus.cc:729] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62dfbee4bcb34ff8bdf49b034594544b, State: Initialized, Role: FOLLOWER
I20260923 22:08:00.794449 7077 consensus_queue.cc:260] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } }
I20260923 22:08:00.794529 7076 consensus_queue.cc:260] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } }
I20260923 22:08:00.794705 7077 ts_tablet_manager.cc:1428] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:08:00.794741 7076 ts_tablet_manager.cc:1428] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:08:00.794734 6930 heartbeater.cc:503] Master 127.1.244.254:38183 was elected leader, sending a full tablet report...
I20260923 22:08:00.794862 7061 heartbeater.cc:503] Master 127.1.244.254:38183 was elected leader, sending a full tablet report...
W20260923 22:08:00.796173 6800 tablet.cc:2369] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:08:00.907946 6931 tablet.cc:2369] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:00.998428 7082 raft_consensus.cc:482] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:00.998555 7082 raft_consensus.cc:504] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } }
I20260923 22:08:00.998951 7082 leader_election.cc:288] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66779f1f640042ab804ed2e6ee641d23 (127.1.244.193:42843), 62dfbee4bcb34ff8bdf49b034594544b (127.1.244.195:43049)
I20260923 22:08:01.000347 6932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.999187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42843 (local address 127.1.244.194:39439)
0923 22:08:00.999253 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.999269 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.999333 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.999490 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.999492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.999494 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:00.999998 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:00.999998 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:01.000005 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:01.000015 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:01.000204 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:01.000205 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:01.000206 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.000251 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.000266 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:08:01.000538 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.999297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:39439 (local address 127.1.244.193:42843)
0923 22:08:00.999346 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.999348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.999362 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:08:00.999395 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.999396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.999402 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.999471 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.000097 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.000099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.000099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:01.000103 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.000120 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:01.000125 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.000177 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.000455 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.000470 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:08:01.000593 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.999361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:45273 (local address 127.1.244.195:43049)
0923 22:08:00.999406 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.999408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.999471 (+ 63us) server_negotiation.cc:407] Connection header received
0923 22:08:00.999516 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.999517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.999522 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.999624 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.000310 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.000312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.000312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:01.000316 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.000331 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:01.000335 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.000391 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.000506 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.000522 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:08:01.000784 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.999187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:43049 (local address 127.1.244.194:45273)
0923 22:08:00.999397 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.999407 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.999458 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.999639 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.999640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.999642 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:01.000192 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:01.000193 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:01.000201 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:01.000215 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:01.000396 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:01.000397 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:01.000414 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.000683 (+ 269us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.000697 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":22,"thread_start_us":114,"threads_started":1}
I20260923 22:08:01.000873 6753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a94a078d770492eba48eec4637e3242" candidate_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66779f1f640042ab804ed2e6ee641d23" is_pre_election: true
I20260923 22:08:01.001037 6753 raft_consensus.cc:2454] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45973c9ee3bd45f599c65f7c205e6ab4 in term 0.
I20260923 22:08:01.001060 7015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a94a078d770492eba48eec4637e3242" candidate_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62dfbee4bcb34ff8bdf49b034594544b" is_pre_election: true
I20260923 22:08:01.001185 7015 raft_consensus.cc:2454] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45973c9ee3bd45f599c65f7c205e6ab4 in term 0.
I20260923 22:08:01.001278 6818 leader_election.cc:302] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45973c9ee3bd45f599c65f7c205e6ab4, 66779f1f640042ab804ed2e6ee641d23; no voters:
I20260923 22:08:01.001416 7082 raft_consensus.cc:2781] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:01.001505 7082 raft_consensus.cc:482] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:01.001542 7082 raft_consensus.cc:3037] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.002250 7082 raft_consensus.cc:504] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } }
I20260923 22:08:01.002398 7082 leader_election.cc:288] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 66779f1f640042ab804ed2e6ee641d23 (127.1.244.193:42843), 62dfbee4bcb34ff8bdf49b034594544b (127.1.244.195:43049)
I20260923 22:08:01.002557 7015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a94a078d770492eba48eec4637e3242" candidate_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62dfbee4bcb34ff8bdf49b034594544b"
I20260923 22:08:01.002550 6753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a94a078d770492eba48eec4637e3242" candidate_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66779f1f640042ab804ed2e6ee641d23"
I20260923 22:08:01.002635 6753 raft_consensus.cc:3037] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.002633 7015 raft_consensus.cc:3037] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.003513 6753 raft_consensus.cc:2454] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45973c9ee3bd45f599c65f7c205e6ab4 in term 1.
I20260923 22:08:01.003515 7015 raft_consensus.cc:2454] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45973c9ee3bd45f599c65f7c205e6ab4 in term 1.
I20260923 22:08:01.003710 6818 leader_election.cc:302] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45973c9ee3bd45f599c65f7c205e6ab4, 66779f1f640042ab804ed2e6ee641d23; no voters:
I20260923 22:08:01.003834 7082 raft_consensus.cc:2781] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:01.003988 7082 raft_consensus.cc:686] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 1 LEADER]: Becoming Leader. State: Replica: 45973c9ee3bd45f599c65f7c205e6ab4, State: Running, Role: LEADER
I20260923 22:08:01.004091 7082 consensus_queue.cc:237] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } }
I20260923 22:08:01.004921 6610 catalog_manager.cc:5495] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45973c9ee3bd45f599c65f7c205e6ab4 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45973c9ee3bd45f599c65f7c205e6ab4" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 33687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:01.012463 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.011430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.1.244.194:33687)
0923 22:08:01.011488 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.011490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.011498 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:01.011593 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.011594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.011608 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.011670 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.012218 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.012219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.012220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:01.012223 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.012254 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:01.012259 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.012317 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.012355 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.012365 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:08:01.014117 6753 raft_consensus.cc:1261] T 2a94a078d770492eba48eec4637e3242 P 66779f1f640042ab804ed2e6ee641d23 [term 1 FOLLOWER]: Refusing update from remote peer 45973c9ee3bd45f599c65f7c205e6ab4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:01.014443 7015 raft_consensus.cc:1261] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [term 1 FOLLOWER]: Refusing update from remote peer 45973c9ee3bd45f599c65f7c205e6ab4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260923 22:08:01.014520 7062 tablet.cc:2369] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:01.014487 7085 consensus_queue.cc:1035] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66779f1f640042ab804ed2e6ee641d23" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:01.014705 7082 consensus_queue.cc:1035] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62dfbee4bcb34ff8bdf49b034594544b" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:01.017098 7089 mvcc.cc:205] Tried to move back new op lower bound from 7332664447031197696 to 7332664446993317888. Current Snapshot: MvccSnapshot[applied={T|T < 7332664447031197696}]
I20260923 22:08:01.017555 7090 mvcc.cc:205] Tried to move back new op lower bound from 7332664447031197696 to 7332664446993317888. Current Snapshot: MvccSnapshot[applied={T|T < 7332664447031197696}]
I20260923 22:08:01.019696 6611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:01.019791 6611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:01.020522 6611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P daf5286a9e084cec95f0fd86855de045: Sending DeleteTablet for 3 replicas of tablet 2a94a078d770492eba48eec4637e3242
I20260923 22:08:01.020830 6995 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a94a078d770492eba48eec4637e3242 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:01 UTC) from {username='slave'} at 127.0.0.1:41448
I20260923 22:08:01.020833 6733 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a94a078d770492eba48eec4637e3242 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:01 UTC) from {username='slave'} at 127.0.0.1:42148
I20260923 22:08:01.021078 6864 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a94a078d770492eba48eec4637e3242 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:01 UTC) from {username='slave'} at 127.0.0.1:54644
I20260923 22:08:01.021095 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 6671
I20260923 22:08:01.021698 7099 tablet_replica.cc:330] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: stopping tablet replica
I20260923 22:08:01.021793 7099 raft_consensus.cc:2229] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:01.021906 7099 raft_consensus.cc:2258] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:01.022357 7099 ts_tablet_manager.cc:1905] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:01.021044 7097 tablet_replica.cc:330] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: stopping tablet replica
I20260923 22:08:01.022543 7097 raft_consensus.cc:2229] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:01.022686 7097 raft_consensus.cc:2258] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:01.023972 7099 ts_tablet_manager.cc:1918] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:01.024047 7099 log.cc:1196] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/2a94a078d770492eba48eec4637e3242
I20260923 22:08:01.024340 7099 ts_tablet_manager.cc:1939] T 2a94a078d770492eba48eec4637e3242 P 45973c9ee3bd45f599c65f7c205e6ab4: Deleting consensus metadata
I20260923 22:08:01.024327 7097 ts_tablet_manager.cc:1905] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:01.024981 6596 catalog_manager.cc:4841] TS 45973c9ee3bd45f599c65f7c205e6ab4 (127.1.244.194:33687): tablet 2a94a078d770492eba48eec4637e3242 (table test-table [id=a4600d522522425281bb59b35372a72d]) successfully deleted
I20260923 22:08:01.025669 7097 ts_tablet_manager.cc:1918] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:01.025727 7097 log.cc:1196] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/2a94a078d770492eba48eec4637e3242
I20260923 22:08:01.025977 7097 ts_tablet_manager.cc:1939] T 2a94a078d770492eba48eec4637e3242 P 62dfbee4bcb34ff8bdf49b034594544b: Deleting consensus metadata
I20260923 22:08:01.026350 6597 catalog_manager.cc:4841] TS 62dfbee4bcb34ff8bdf49b034594544b (127.1.244.195:43049): tablet 2a94a078d770492eba48eec4637e3242 (table test-table [id=a4600d522522425281bb59b35372a72d]) successfully deleted
W20260923 22:08:01.027609 6596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:42843 (error 108)
I20260923 22:08:01.027933 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 6802
I20260923 22:08:01.028090 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.027953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42843 (local address 127.0.0.1:42150)
0923 22:08:01.028012 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.028058 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 22:08:01.028504 6596 catalog_manager.cc:4584] TS 66779f1f640042ab804ed2e6ee641d23 (127.1.244.193:42843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a94a078d770492eba48eec4637e3242: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42843: connect: Connection refused (error 111)
I20260923 22:08:01.033439 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 6933
I20260923 22:08:01.037693 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 6580
2026-09-23T22:08:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (763 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260923 22:08:01.056864 2003 mini_oidc.cc:152] Starting JWKS server
I20260923 22:08:01.058218 2003 webserver.cc:460] Webserver started at https://127.0.0.1:39355/ using document root <none> and password file <none>
I20260923 22:08:01.058344 2003 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:08:01.058511 2003 webserver.cc:460] Webserver started at http://127.0.0.1:46033/ using document root <none> and password file <none>
2026-09-23T22:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:01Z Disabled control of system clock
I20260923 22:08:01.069721 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:34891
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:40931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:34891
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39355/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/testchainca.cert with env {}
W20260923 22:08:01.149756 7106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:01.149955 7106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:01.149976 7106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:01.151156 7106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:01.151196 7106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:01.151209 7106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:01.151222 7106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:01.151304 7106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:08:01.151327 7106 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:08:01.151355 7106 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39355/jwks.json
I20260923 22:08:01.152989 7106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:34891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39355/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:01.153198 7106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:01.153405 7106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:01.158165 7112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.158298 7106 server_base.cc:1034] running on GCE node
W20260923 22:08:01.158200 7114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.158243 7111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.158583 7106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:01.158927 7106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:01.160087 7106 hybrid_clock.cc:648] HybridClock initialized: now 1790201281160075 us; error 43 us; skew 500 ppm
I20260923 22:08:01.161332 7106 webserver.cc:460] Webserver started at http://127.1.244.254:39543/ using document root <none> and password file <none>
I20260923 22:08:01.161574 7106 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:01.161648 7106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:01.161804 7106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:01.162827 7106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "60c3c7de351b4a899be6e51d8995a0d9"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "8f88e43aa60942811a18270c7c2ccb89"
server_key_iv: "cb0d125e9f652a1ed1c18d76627c628c"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.163182 7106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "60c3c7de351b4a899be6e51d8995a0d9"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "8f88e43aa60942811a18270c7c2ccb89"
server_key_iv: "cb0d125e9f652a1ed1c18d76627c628c"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.164529 7106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:01.165235 7120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.165390 7106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:01.165495 7106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "60c3c7de351b4a899be6e51d8995a0d9"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "8f88e43aa60942811a18270c7c2ccb89"
server_key_iv: "cb0d125e9f652a1ed1c18d76627c628c"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.165570 7106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:01.178823 7106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:01.179150 7106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:01.179309 7106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:01.183270 7106 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:34891
I20260923 22:08:01.183305 7172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:34891 every 8 connection(s)
I20260923 22:08:01.183712 7106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:01.184625 7173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.185189 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 7106
I20260923 22:08:01.185292 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:01.185487 2003 external_mini_cluster.cc:1428] Setting key a5a2ce108c2368ab30320d265606e1a3
I20260923 22:08:01.187222 7173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9: Bootstrap starting.
I20260923 22:08:01.187939 7173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.188257 7173 log.cc:824] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:01.189052 7173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9: No bootstrap required, opened a new log
I20260923 22:08:01.191222 7173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c3c7de351b4a899be6e51d8995a0d9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34891 } }
I20260923 22:08:01.191401 7173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.191440 7173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c3c7de351b4a899be6e51d8995a0d9, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.191613 7173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c3c7de351b4a899be6e51d8995a0d9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34891 } }
I20260923 22:08:01.191696 7173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:01.191727 7173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:01.191776 7173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.192315 7173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c3c7de351b4a899be6e51d8995a0d9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34891 } }
I20260923 22:08:01.192418 7173 leader_election.cc:302] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60c3c7de351b4a899be6e51d8995a0d9; no voters:
I20260923 22:08:01.192574 7173 leader_election.cc:288] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:01.192652 7178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:01.192811 7178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [term 1 LEADER]: Becoming Leader. State: Replica: 60c3c7de351b4a899be6e51d8995a0d9, State: Running, Role: LEADER
I20260923 22:08:01.192854 7173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:01.192893 7178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c3c7de351b4a899be6e51d8995a0d9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34891 } }
I20260923 22:08:01.193207 7178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c3c7de351b4a899be6e51d8995a0d9. Latest consensus state: current_term: 1 leader_uuid: "60c3c7de351b4a899be6e51d8995a0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c3c7de351b4a899be6e51d8995a0d9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34891 } } }
I20260923 22:08:01.193306 7178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:01.193218 7179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60c3c7de351b4a899be6e51d8995a0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c3c7de351b4a899be6e51d8995a0d9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34891 } } }
I20260923 22:08:01.193441 7179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:01.193244 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.186066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.1.244.254:34891)
0923 22:08:01.186265 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.186274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.187478 (+ 1204us) server_negotiation.cc:407] Connection header received
0923 22:08:01.188134 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.188142 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.188228 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.188383 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.188663 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.190030 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.190786 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.191413 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.191540 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.192322 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.192326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.192327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:01.192539 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.192582 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:01.192597 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.192669 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.192938 (+ 269us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.193053 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260923 22:08:01.194082 7184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:01.194404 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:08:01.195304 7184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:01.196805 7184 catalog_manager.cc:1347] Generated new cluster ID: b839b0c8f14a443f85c0aeed236b80bc
I20260923 22:08:01.196866 7184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:01.203969 7184 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:01.204485 7184 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:01.212772 7184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60c3c7de351b4a899be6e51d8995a0d9: Generated new TSK 0
I20260923 22:08:01.212956 7184 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 22:08:01.240434 7203 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260923 22:08:01.240525 7203 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 22:08:01.240734 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.225923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.1.244.254:34891)
0923 22:08:01.225969 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.225971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.226096 (+ 125us) server_negotiation.cc:407] Connection header received
0923 22:08:01.226194 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.226195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.226204 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.226260 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:08:01.226416 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.226897 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.228033 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.228223 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.228275 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.228500 (+ 225us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:08:01.240518 (+ 12018us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 22:08:01.240662 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49198: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260923 22:08:01.240814 7176 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49198: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260923 22:08:01.241080 7202 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:01.225820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34891 (local address 127.0.0.1:49198)
0923 22:08:01.226025 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.226037 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.226107 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.226272 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.226273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.226275 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:08:01.226356 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.226358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.226969 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.226970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.227958 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.227960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.228300 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.228301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.228375 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.228398 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:08:01.240685 (+ 12287us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 22:08:01.240788 (+ 103us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.244.254:34891: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.244.254:34891: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39355/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260923 22:08:01.241621 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 7106
2026-09-23T22:08:01Z chronyd exiting
I20260923 22:08:01.277860 2003 test_util.cc:175] -----------------------------------------------
I20260923 22:08:01.277961 2003 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201256447402-2003-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (222 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260923 22:08:01.278702 2003 mini_oidc.cc:152] Starting JWKS server
I20260923 22:08:01.279383 2003 webserver.cc:460] Webserver started at https://127.0.0.1:35971/ using document root <none> and password file <none>
I20260923 22:08:01.279452 2003 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:08:01.279703 2003 webserver.cc:460] Webserver started at http://127.0.0.1:43003/ using document root <none> and password file <none>
2026-09-23T22:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:01Z Disabled control of system clock
I20260923 22:08:01.287070 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36137
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:42163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:36137
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35971/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/testchainca.cert with env {}
W20260923 22:08:01.365568 7209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:01.365819 7209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:01.365861 7209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:01.367043 7209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:01.367116 7209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:01.367138 7209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:01.367161 7209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:01.367190 7209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:08:01.367213 7209 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:08:01.367241 7209 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35971/jwks.json
I20260923 22:08:01.368813 7209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:36137
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35971/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:01.369091 7209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:01.369313 7209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:01.373898 7217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.373952 7214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.373915 7215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.374169 7209 server_base.cc:1034] running on GCE node
I20260923 22:08:01.374416 7209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:01.374650 7209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:01.375798 7209 hybrid_clock.cc:648] HybridClock initialized: now 1790201281375781 us; error 28 us; skew 500 ppm
I20260923 22:08:01.377152 7209 webserver.cc:460] Webserver started at http://127.1.244.254:37403/ using document root <none> and password file <none>
I20260923 22:08:01.377378 7209 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:01.377429 7209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:01.377519 7209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:01.378605 7209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "2796afbe381247049ba50dba0930d32e"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "e4d2b8b2a3bebe9a8b02827114379943"
server_key_iv: "29f8bb03cf3cf6fcc4dc2e756325876f"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.378921 7209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "2796afbe381247049ba50dba0930d32e"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "e4d2b8b2a3bebe9a8b02827114379943"
server_key_iv: "29f8bb03cf3cf6fcc4dc2e756325876f"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.380411 7209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:01.381145 7223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.381349 7209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:01.381428 7209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "2796afbe381247049ba50dba0930d32e"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "e4d2b8b2a3bebe9a8b02827114379943"
server_key_iv: "29f8bb03cf3cf6fcc4dc2e756325876f"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.381480 7209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:01.401821 7209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:01.402110 7209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:01.402258 7209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:01.405953 7209 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:36137
I20260923 22:08:01.405987 7275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:36137 every 8 connection(s)
I20260923 22:08:01.406317 7209 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:01.407255 7276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.409507 7276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e: Bootstrap starting.
I20260923 22:08:01.410066 7276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.410359 7276 log.cc:824] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:01.411022 7276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e: No bootstrap required, opened a new log
I20260923 22:08:01.411432 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 7209
I20260923 22:08:01.411496 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:01.411697 2003 external_mini_cluster.cc:1428] Setting key cef89298899494b0a128a85b3e1db369
I20260923 22:08:01.413251 7276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2796afbe381247049ba50dba0930d32e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36137 } }
I20260923 22:08:01.413434 7276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.413470 7276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2796afbe381247049ba50dba0930d32e, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.413621 7276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2796afbe381247049ba50dba0930d32e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36137 } }
I20260923 22:08:01.413724 7276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:01.413756 7276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:01.413795 7276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.414541 7276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2796afbe381247049ba50dba0930d32e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36137 } }
I20260923 22:08:01.414650 7276 leader_election.cc:302] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2796afbe381247049ba50dba0930d32e; no voters:
I20260923 22:08:01.414803 7276 leader_election.cc:288] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:01.414842 7281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:01.414997 7281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [term 1 LEADER]: Becoming Leader. State: Replica: 2796afbe381247049ba50dba0930d32e, State: Running, Role: LEADER
I20260923 22:08:01.415072 7276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:01.415119 7281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2796afbe381247049ba50dba0930d32e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36137 } }
I20260923 22:08:01.415511 7281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2796afbe381247049ba50dba0930d32e. Latest consensus state: current_term: 1 leader_uuid: "2796afbe381247049ba50dba0930d32e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2796afbe381247049ba50dba0930d32e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36137 } } }
I20260923 22:08:01.415519 7283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2796afbe381247049ba50dba0930d32e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2796afbe381247049ba50dba0930d32e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36137 } } }
I20260923 22:08:01.415705 7283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:01.415900 7281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:01.416491 7290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:01.416980 7290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:01.418859 7280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.412200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.1.244.254:36137)
0923 22:08:01.412428 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.412437 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.412470 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:08:01.413213 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.413221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.413302 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.413456 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.413723 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.415478 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.416603 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.417260 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.417379 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.418104 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.418108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.418109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:01.418303 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.418343 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:01.418357 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.418426 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.418550 (+ 124us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.418639 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260923 22:08:01.419070 7290 catalog_manager.cc:1347] Generated new cluster ID: 6a6b3ff8d1714defb01fc0b02b951d2f
I20260923 22:08:01.419152 7290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:01.420008 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:08:01.448160 7290 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:01.448690 7290 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:01.451134 7290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2796afbe381247049ba50dba0930d32e: Generated new TSK 0
I20260923 22:08:01.451308 7290 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 22:08:01.473127 7306 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260923 22:08:01.473205 7306 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 22:08:01.473515 7280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.459438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.1.244.254:36137)
0923 22:08:01.459484 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.459487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.459602 (+ 115us) server_negotiation.cc:407] Connection header received
0923 22:08:01.459683 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.459684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.459694 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.459757 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:08:01.459899 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.460351 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.461480 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.461660 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.461708 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.461894 (+ 186us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:08:01.473340 (+ 11446us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 22:08:01.473467 (+ 127us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260923 22:08:01.473593 7280 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260923 22:08:01.473667 7305 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:01.459328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36137 (local address 127.0.0.1:51430)
0923 22:08:01.459513 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.459524 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.459612 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.459770 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.459771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.459773 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:08:01.459832 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.459835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.460445 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.460446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.461396 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.461399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.461738 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.461739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.461799 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.461804 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:08:01.473458 (+ 11654us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 22:08:01.473571 (+ 113us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.244.254:36137: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":122,"threads_started":1}
I20260923 22:08:01.474045 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 7209
2026-09-23T22:08:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (233 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260923 22:08:01.512001 2003 mini_oidc.cc:152] Starting JWKS server
I20260923 22:08:01.512612 2003 webserver.cc:460] Webserver started at https://127.0.0.1:33329/ using document root <none> and password file <none>
I20260923 22:08:01.512651 2003 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:08:01.512763 2003 webserver.cc:460] Webserver started at http://127.0.0.1:37501/ using document root <none> and password file <none>
2026-09-23T22:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:01Z Disabled control of system clock
I20260923 22:08:01.520736 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37655
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:37655
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33329/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 22:08:01.599402 7312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:01.599634 7312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:01.599663 7312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:01.600841 7312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:01.600880 7312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:01.600893 7312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:01.600906 7312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:01.600919 7312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:08:01.600930 7312 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:08:01.600942 7312 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33329/jwks.json
I20260923 22:08:01.602461 7312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:37655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33329/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:01.602658 7312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:01.602881 7312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:01.607436 7320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.607421 7318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.607525 7312 server_base.cc:1034] running on GCE node
W20260923 22:08:01.607456 7317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.607956 7312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:01.608264 7312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:01.609431 7312 hybrid_clock.cc:648] HybridClock initialized: now 1790201281609424 us; error 40 us; skew 500 ppm
I20260923 22:08:01.610636 7312 webserver.cc:460] Webserver started at http://127.1.244.254:34761/ using document root <none> and password file <none>
I20260923 22:08:01.610893 7312 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:01.610962 7312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:01.611111 7312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:01.612217 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "fd1450550e0f46f1b0b1ef2745bbc999"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "8d2fefb20203cb38f0c8a214b0020da5"
server_key_iv: "595ce6d0b66db5c7a1a096e5c531bb68"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.612571 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "fd1450550e0f46f1b0b1ef2745bbc999"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "8d2fefb20203cb38f0c8a214b0020da5"
server_key_iv: "595ce6d0b66db5c7a1a096e5c531bb68"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.613852 7312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 22:08:01.614651 7326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.614845 7312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:01.614951 7312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "fd1450550e0f46f1b0b1ef2745bbc999"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "8d2fefb20203cb38f0c8a214b0020da5"
server_key_iv: "595ce6d0b66db5c7a1a096e5c531bb68"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.615027 7312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:01.631490 7312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:01.631814 7312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:01.631966 7312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:01.635686 7312 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:37655
I20260923 22:08:01.635718 7378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:37655 every 8 connection(s)
I20260923 22:08:01.636051 7312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:01.636735 7379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.639318 7379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999: Bootstrap starting.
I20260923 22:08:01.639957 7379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.640247 7379 log.cc:824] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:01.640911 7379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999: No bootstrap required, opened a new log
I20260923 22:08:01.642480 7379 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1450550e0f46f1b0b1ef2745bbc999" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37655 } }
I20260923 22:08:01.642613 7379 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.642663 7379 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd1450550e0f46f1b0b1ef2745bbc999, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.642820 7379 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1450550e0f46f1b0b1ef2745bbc999" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37655 } }
I20260923 22:08:01.642884 7379 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:01.642925 7379 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:01.642977 7379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.643492 7379 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1450550e0f46f1b0b1ef2745bbc999" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37655 } }
I20260923 22:08:01.643631 7379 leader_election.cc:302] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd1450550e0f46f1b0b1ef2745bbc999; no voters:
I20260923 22:08:01.643810 7379 leader_election.cc:288] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:01.643836 7382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:01.644024 7382 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [term 1 LEADER]: Becoming Leader. State: Replica: fd1450550e0f46f1b0b1ef2745bbc999, State: Running, Role: LEADER
I20260923 22:08:01.644052 7379 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:01.644146 7382 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1450550e0f46f1b0b1ef2745bbc999" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37655 } }
I20260923 22:08:01.644506 7383 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd1450550e0f46f1b0b1ef2745bbc999" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1450550e0f46f1b0b1ef2745bbc999" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37655 } } }
I20260923 22:08:01.644657 7383 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:01.644870 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 7312
I20260923 22:08:01.644972 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:01.645182 2003 external_mini_cluster.cc:1428] Setting key a705c5982829e112dae2883e9a28278f
I20260923 22:08:01.644644 7384 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd1450550e0f46f1b0b1ef2745bbc999. Latest consensus state: current_term: 1 leader_uuid: "fd1450550e0f46f1b0b1ef2745bbc999" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1450550e0f46f1b0b1ef2745bbc999" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37655 } } }
I20260923 22:08:01.645324 7384 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999 [sys.catalog]: This master's current role is: LEADER
W20260923 22:08:01.645979 7398 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:08:01.646047 7398 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:08:01.646152 7391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:01.646341 7391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:01.649287 7391 catalog_manager.cc:1347] Generated new cluster ID: ace6c310b3774259a1791daa59ae5761
I20260923 22:08:01.649345 7391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:01.651849 7400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.645627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49384 (local address 127.1.244.254:37655)
0923 22:08:01.645884 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.645894 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.645929 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:08:01.646049 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.646055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.646123 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.647044 (+ 921us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.647318 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.648828 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.649803 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.650299 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.650448 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.651165 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.651169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.651169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:01.651362 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.651399 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:01.651410 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.651482 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.651603 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.651684 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":69,"threads_started":1}
I20260923 22:08:01.652782 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:08:01.667331 7391 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:01.667909 7391 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:01.672837 7391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd1450550e0f46f1b0b1ef2745bbc999: Generated new TSK 0
I20260923 22:08:01.673010 7391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:01.684531 7400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.681152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49394 (local address 127.1.244.254:37655)
0923 22:08:01.681224 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.681227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.681260 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:08:01.681362 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.681363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.681387 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.681468 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:08:01.681604 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.682122 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.682909 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.683114 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.683205 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.683387 (+ 182us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:08:01.684337 (+ 950us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33329/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 22:08:01.684469 (+ 132us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49394: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33329/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260923 22:08:01.684582 7400 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49394: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33329/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260923 22:08:01.684665 7408 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:01.681053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37655 (local address 127.0.0.1:49394)
0923 22:08:01.681199 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.681211 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.681278 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.681468 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.681469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.681472 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:08:01.681538 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.681539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.682222 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.682223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.682827 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.682829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.683209 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.683210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.683272 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.683276 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:08:01.684447 (+ 1171us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33329/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 22:08:01.684553 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.244.254:37655: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33329/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260923 22:08:01.684924 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 7312
E20260923 22:08:01.695447 7409 webserver.cc:533] Webserver: sslize failed:
E20260923 22:08:01.695508 7409 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-23T22:08:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (210 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-23T22:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:01Z Disabled control of system clock
I20260923 22:08:01.730262 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38255
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38255
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/testchainca.cert with env {}
W20260923 22:08:01.809134 7413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:01.809373 7413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:01.809412 7413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:01.810577 7413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:01.810644 7413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:01.810667 7413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:01.810688 7413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:01.810719 7413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:08:01.810743 7413 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:08:01.810779 7413 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260923 22:08:01.812302 7413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38255
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:01.812579 7413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:01.812805 7413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:01.817577 7419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.817776 7413 server_base.cc:1034] running on GCE node
W20260923 22:08:01.817551 7421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.817557 7418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.818097 7413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:01.818384 7413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:01.819587 7413 hybrid_clock.cc:648] HybridClock initialized: now 1790201281819576 us; error 35 us; skew 500 ppm
I20260923 22:08:01.820803 7413 webserver.cc:460] Webserver started at http://127.1.244.254:46577/ using document root <none> and password file <none>
I20260923 22:08:01.821048 7413 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:01.821115 7413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:01.821245 7413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:01.822283 7413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "1dc65221badf4d418985ff637d3c44e8"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "35fef8031cd71822270cc32800da440d"
server_key_iv: "221e5cf3cb06b314653f597b45a7dcca"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.822705 7413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "1dc65221badf4d418985ff637d3c44e8"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "35fef8031cd71822270cc32800da440d"
server_key_iv: "221e5cf3cb06b314653f597b45a7dcca"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.824116 7413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:01.824800 7427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.824971 7413 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:01.825069 7413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "1dc65221badf4d418985ff637d3c44e8"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-b5jz"
server_key: "35fef8031cd71822270cc32800da440d"
server_key_iv: "221e5cf3cb06b314653f597b45a7dcca"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.825147 7413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:01.839481 7413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:01.839794 7413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:01.839938 7413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:01.843690 7413 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:38255
I20260923 22:08:01.843721 7479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:38255 every 8 connection(s)
I20260923 22:08:01.844045 7413 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:01.844684 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 7413
I20260923 22:08:01.844781 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:01.844995 2003 external_mini_cluster.cc:1428] Setting key 1fd4d22936fd32080d26e9022af06e27
I20260923 22:08:01.845209 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.848032 7480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8: Bootstrap starting.
I20260923 22:08:01.848601 7480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.848910 7480 log.cc:824] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:01.849637 7480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8: No bootstrap required, opened a new log
I20260923 22:08:01.850519 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.845538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.1.244.254:38255)
0923 22:08:01.845757 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.845765 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.845792 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:08:01.846323 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.846330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.846391 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.846546 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.846765 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.847706 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.848558 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.849073 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.849220 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.849908 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.849912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.849913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:01.850081 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.850111 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:01.850122 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.850194 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.850268 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.850357 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260923 22:08:01.851533 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:08:01.851837 7480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dc65221badf4d418985ff637d3c44e8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38255 } }
I20260923 22:08:01.851986 7480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.852030 7480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dc65221badf4d418985ff637d3c44e8, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.852144 7480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dc65221badf4d418985ff637d3c44e8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38255 } }
I20260923 22:08:01.852237 7480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:01.852278 7480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:01.852330 7480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.852895 7480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dc65221badf4d418985ff637d3c44e8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38255 } }
I20260923 22:08:01.852979 7480 leader_election.cc:302] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dc65221badf4d418985ff637d3c44e8; no voters:
I20260923 22:08:01.853092 7480 leader_election.cc:288] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:01.853158 7485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:01.853327 7485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1dc65221badf4d418985ff637d3c44e8, State: Running, Role: LEADER
I20260923 22:08:01.853338 7480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:01.853433 7485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dc65221badf4d418985ff637d3c44e8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38255 } }
I20260923 22:08:01.853845 7487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dc65221badf4d418985ff637d3c44e8. Latest consensus state: current_term: 1 leader_uuid: "1dc65221badf4d418985ff637d3c44e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dc65221badf4d418985ff637d3c44e8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38255 } } }
I20260923 22:08:01.853842 7486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dc65221badf4d418985ff637d3c44e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dc65221badf4d418985ff637d3c44e8" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38255 } } }
I20260923 22:08:01.853936 7487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:01.853942 7486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:01.854233 7493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:01.855173 7493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:01.856622 7493 catalog_manager.cc:1347] Generated new cluster ID: a522662bb3f24115b1b18c47575d7dcc
I20260923 22:08:01.856676 7493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:01.873241 7493 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:01.873759 7493 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:01.880458 7493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dc65221badf4d418985ff637d3c44e8: Generated new TSK 0
I20260923 22:08:01.880652 7493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:02.349623 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.888850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58184 (local address 127.1.244.254:38255)
0923 22:08:01.888925 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.888927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.888981 (+ 54us) server_negotiation.cc:407] Connection header received
0923 22:08:01.889046 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.889047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.889059 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.889134 (+ 75us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:08:01.889238 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.890035 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.890745 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.891069 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.891140 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.891280 (+ 140us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:08:02.349312 (+458032us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 22:08:02.349543 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58184: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":49}
W20260923 22:08:02.349691 7483 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58184: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260923 22:08:02.349705 7509 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:01.888755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38255 (local address 127.0.0.1:58184)
0923 22:08:01.888929 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.888939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.888993 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.889137 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.889138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.889145 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:08:01.889191 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.889192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.890100 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.890101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.890666 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.890668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.891144 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.891145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.891188 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.891193 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:08:02.349477 (+458284us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 22:08:02.349602 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.244.254:38255: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260923 22:08:02.350050 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 7413
2026-09-23T22:08:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (645 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:02 dist-test-slave-b5jz krb5kdc[7521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:02 dist-test-slave-b5jz krb5kdc[7521](info): set up 2 sockets
Sep 23 22:08:02 dist-test-slave-b5jz krb5kdc[7521](info): commencing operation
krb5kdc: starting...
W20260923 22:08:03.599838 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:08:03 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:08:03.713574 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40375
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:40375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:03.794250 7534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:03.794469 7534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:03.794492 7534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:03.795616 7534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:03.795652 7534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:03.795666 7534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:03.795679 7534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:03.795691 7534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:03.797091 7534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:40375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:03.797309 7534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:03.797474 7534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:03.802191 7539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:03.802208 7540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:03.802246 7542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:03.802521 7534 server_base.cc:1034] running on GCE node
I20260923 22:08:03.802790 7534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:03.803083 7534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:03.804250 7534 hybrid_clock.cc:648] HybridClock initialized: now 1790201283804236 us; error 32 us; skew 500 ppm
Sep 23 22:08:03 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201283, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:03.806790 7534 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:08:03.807272 7534 webserver.cc:460] Webserver started at http://127.1.244.254:35685/ using document root <none> and password file <none>
I20260923 22:08:03.807512 7534 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:03.807623 7534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:03.807765 7534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:03.808837 7534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "97658fa45631449f8238efea3248bb61"
format_stamp: "Formatted at 2026-09-23 22:08:03 on dist-test-slave-b5jz"
server_key: "701056458dbe6da965c91f32ab1da558"
server_key_iv: "334a1394bfd7cf9ab5e12ea464db35ad"
server_key_version: "encryptionkey@0"
I20260923 22:08:03.809190 7534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "97658fa45631449f8238efea3248bb61"
format_stamp: "Formatted at 2026-09-23 22:08:03 on dist-test-slave-b5jz"
server_key: "701056458dbe6da965c91f32ab1da558"
server_key_iv: "334a1394bfd7cf9ab5e12ea464db35ad"
server_key_version: "encryptionkey@0"
I20260923 22:08:03.810470 7534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:03.811200 7549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:03.811388 7534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:03.811479 7534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "97658fa45631449f8238efea3248bb61"
format_stamp: "Formatted at 2026-09-23 22:08:03 on dist-test-slave-b5jz"
server_key: "701056458dbe6da965c91f32ab1da558"
server_key_iv: "334a1394bfd7cf9ab5e12ea464db35ad"
server_key_version: "encryptionkey@0"
I20260923 22:08:03.811583 7534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:03.825373 7534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:03.827231 7534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:03.827376 7534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:03.831200 7534 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:40375
I20260923 22:08:03.831265 7601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:40375 every 8 connection(s)
I20260923 22:08:03.831661 7534 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:03.832497 7602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:03.834707 7602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61: Bootstrap starting.
I20260923 22:08:03.835259 7602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:03.835592 7602 log.cc:824] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:03.836246 7602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61: No bootstrap required, opened a new log
I20260923 22:08:03.837823 7602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97658fa45631449f8238efea3248bb61" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40375 } }
I20260923 22:08:03.837954 7602 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:03.837976 7602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97658fa45631449f8238efea3248bb61, State: Initialized, Role: FOLLOWER
I20260923 22:08:03.838140 7602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97658fa45631449f8238efea3248bb61" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40375 } }
I20260923 22:08:03.838239 7602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:03.838279 7602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:03.838351 7602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:03.838397 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 7534
I20260923 22:08:03.838532 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:03.838724 2003 external_mini_cluster.cc:1428] Setting key 5a3a7c6fa79447834fe3351881378f72
I20260923 22:08:03.838989 7602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97658fa45631449f8238efea3248bb61" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40375 } }
I20260923 22:08:03.839110 7602 leader_election.cc:302] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97658fa45631449f8238efea3248bb61; no voters:
I20260923 22:08:03.839313 7602 leader_election.cc:288] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:03.839607 7602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:03.839648 7607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:03.839803 7607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [term 1 LEADER]: Becoming Leader. State: Replica: 97658fa45631449f8238efea3248bb61, State: Running, Role: LEADER
I20260923 22:08:03.839902 7607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97658fa45631449f8238efea3248bb61" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40375 } }
I20260923 22:08:03.840317 7608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97658fa45631449f8238efea3248bb61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97658fa45631449f8238efea3248bb61" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40375 } } }
I20260923 22:08:03.840397 7608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:03.840530 7608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97658fa45631449f8238efea3248bb61. Latest consensus state: current_term: 1 leader_uuid: "97658fa45631449f8238efea3248bb61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97658fa45631449f8238efea3248bb61" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40375 } } }
I20260923 22:08:03.840585 7608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61 [sys.catalog]: This master's current role is: LEADER
W20260923 22:08:03.841387 7623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:08:03.841455 7623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:08:03.841548 7616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:03.841718 7616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:03.843154 7616 catalog_manager.cc:1347] Generated new cluster ID: 69ffc20019804ffdb19c69c661ce3c53
I20260923 22:08:03.843209 7616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:08:03 dist-test-slave-b5jz krb5kdc[7521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:03.849601 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:03.839159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.1.244.254:40375)
0923 22:08:03.839369 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:03.839379 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:03.839425 (+ 46us) server_negotiation.cc:407] Connection header received
0923 22:08:03.839567 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:03.839601 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:03.839666 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:03.840656 (+ 990us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:03.840765 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:03.842346 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:03.843382 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:03.843876 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:03.844014 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:03.846294 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:03.846298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:03.846298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:03.846309 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:03.848093 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:03.848451 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:03.848455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:03.848455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:03.848550 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:03.848765 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:03.848766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:03.848766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:03.848996 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:03.849152 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:03.849260 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 22:08:03.849401 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":107,"threads_started":1}
I20260923 22:08:03.871876 7616 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:03.872447 7616 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:03.881103 7616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97658fa45631449f8238efea3248bb61: Generated new TSK 0
I20260923 22:08:03.881294 7616 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:08:03.912483 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40375
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:04.015946 7628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:04.016145 7628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:04.016168 7628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:04.017271 7628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:04.017343 7628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:04.017390 7628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:04.018793 7628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:04.018981 7628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:04.019152 7628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:04.023876 7636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:04.023882 7633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:04.023890 7634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:04.023977 7628 server_base.cc:1034] running on GCE node
I20260923 22:08:04.024390 7628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:04.024652 7628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:04.025816 7628 hybrid_clock.cc:648] HybridClock initialized: now 1790201284025801 us; error 48 us; skew 500 ppm
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.028240 7628 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:08:04.028688 7628 webserver.cc:460] Webserver started at http://127.1.244.193:41763/ using document root <none> and password file <none>
I20260923 22:08:04.028867 7628 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:04.028908 7628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:04.028992 7628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:04.029955 7628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "7c7c45c0f86c4069953c2ecb7900e810"
format_stamp: "Formatted at 2026-09-23 22:08:04 on dist-test-slave-b5jz"
server_key: "99793ece49a3e93a6451f63a11047616"
server_key_iv: "3f0f70c27810c29251aed8359d2c2be1"
server_key_version: "encryptionkey@0"
I20260923 22:08:04.030247 7628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "7c7c45c0f86c4069953c2ecb7900e810"
format_stamp: "Formatted at 2026-09-23 22:08:04 on dist-test-slave-b5jz"
server_key: "99793ece49a3e93a6451f63a11047616"
server_key_iv: "3f0f70c27810c29251aed8359d2c2be1"
server_key_version: "encryptionkey@0"
I20260923 22:08:04.031447 7628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:04.032431 7643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:04.032622 7628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:04.032738 7628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "7c7c45c0f86c4069953c2ecb7900e810"
format_stamp: "Formatted at 2026-09-23 22:08:04 on dist-test-slave-b5jz"
server_key: "99793ece49a3e93a6451f63a11047616"
server_key_iv: "3f0f70c27810c29251aed8359d2c2be1"
server_key_version: "encryptionkey@0"
I20260923 22:08:04.032814 7628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:04.040979 7628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:04.043011 7628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:04.043169 7628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:04.043402 7628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:04.043753 7628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:04.043805 7628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:04.043846 7628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:04.043872 7628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:04.049873 7628 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:40959
I20260923 22:08:04.049914 7756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:40959 every 8 connection(s)
I20260923 22:08:04.050235 7628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:04.057969 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 7628
I20260923 22:08:04.058087 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:04.058296 2003 external_mini_cluster.cc:1428] Setting key b35314e46389c3104e7bdc103b2e5c3c
I20260923 22:08:04.060118 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:04.050934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:60519 (local address 127.1.244.254:40375)
0923 22:08:04.051002 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:04.051004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:04.051224 (+ 220us) server_negotiation.cc:407] Connection header received
0923 22:08:04.052274 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:04.052275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:04.052292 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:04.052343 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:04.052993 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.053505 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.054510 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.054736 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.054825 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:04.056990 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:04.056992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:04.056992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:04.057001 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:04.058515 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.059066 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.059068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.059068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.059132 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.059404 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.059405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.059405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.059517 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:04.059637 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:04.059927 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 22:08:04.060027 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:08:04.060345 7759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:04.050931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40375 (local address 127.1.244.193:60519)
0923 22:08:04.051101 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:04.051137 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:04.052150 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:04.052419 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:04.052422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:04.052679 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:04.052880 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:04.052884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.053603 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.053605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:04.054416 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:04.054418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.054834 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.054835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:04.054965 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:04.055455 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:04.055463 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:04.056888 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:04.058647 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:04.058654 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:04.058657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:04.058973 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:04.059199 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:04.059200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:04.059201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:04.059324 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:04.059629 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:04.059636 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:04.059785 (+ 149us) client_negotiation.cc:769] Sending connection context
0923 22:08:04.059905 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 22:08:04.060064 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260923 22:08:04.060875 7757 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40375
I20260923 22:08:04.060985 7757 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:04.061162 7757 heartbeater.cc:511] Master 127.1.244.254:40375 requested a full tablet report, sending...
I20260923 22:08:04.061709 7566 ts_manager.cc:194] Registered new tserver with Master: 7c7c45c0f86c4069953c2ecb7900e810 (127.1.244.193:40959)
I20260923 22:08:04.062651 7566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:60519
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:08:04.082840 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40375
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:04.167050 7762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:04.167282 7762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:04.167308 7762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:04.168708 7762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:04.168763 7762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:04.168793 7762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:04.170516 7762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:04.170802 7762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:04.171060 7762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:04.176311 7767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:04.176375 7768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:04.176332 7770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:04.176869 7762 server_base.cc:1034] running on GCE node
I20260923 22:08:04.177040 7762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:04.177294 7762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:04.178474 7762 hybrid_clock.cc:648] HybridClock initialized: now 1790201284178458 us; error 38 us; skew 500 ppm
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.180912 7762 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:08:04.181370 7762 webserver.cc:460] Webserver started at http://127.1.244.194:44465/ using document root <none> and password file <none>
I20260923 22:08:04.181569 7762 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:04.181608 7762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:04.181696 7762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:04.182673 7762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "fba608f4bb3c46a384bba8a83465f610"
format_stamp: "Formatted at 2026-09-23 22:08:04 on dist-test-slave-b5jz"
server_key: "61ee6dc78c24aa56e91442a12f2720dd"
server_key_iv: "c8e914486e48c2e8bca7efacab77feed"
server_key_version: "encryptionkey@0"
I20260923 22:08:04.182964 7762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "fba608f4bb3c46a384bba8a83465f610"
format_stamp: "Formatted at 2026-09-23 22:08:04 on dist-test-slave-b5jz"
server_key: "61ee6dc78c24aa56e91442a12f2720dd"
server_key_iv: "c8e914486e48c2e8bca7efacab77feed"
server_key_version: "encryptionkey@0"
I20260923 22:08:04.184255 7762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:08:04.184942 7777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:04.185141 7762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:04.185226 7762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "fba608f4bb3c46a384bba8a83465f610"
format_stamp: "Formatted at 2026-09-23 22:08:04 on dist-test-slave-b5jz"
server_key: "61ee6dc78c24aa56e91442a12f2720dd"
server_key_iv: "c8e914486e48c2e8bca7efacab77feed"
server_key_version: "encryptionkey@0"
I20260923 22:08:04.185276 7762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:04.197536 7762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:04.199539 7762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:04.199723 7762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:04.199919 7762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:04.200217 7762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:04.200249 7762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:04.200271 7762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:04.200320 7762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:04.205971 7762 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:44477
I20260923 22:08:04.206058 7890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:44477 every 8 connection(s)
I20260923 22:08:04.206370 7762 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:04.207653 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 7762
I20260923 22:08:04.207768 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:04.207978 2003 external_mini_cluster.cc:1428] Setting key 4bc447eda60e807cc33e688b050d0af7
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
I20260923 22:08:04.217161 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:04.207187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:55323 (local address 127.1.244.254:40375)
0923 22:08:04.207243 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:04.207244 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:04.207498 (+ 254us) server_negotiation.cc:407] Connection header received
0923 22:08:04.208693 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:04.208694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:04.208706 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:04.208762 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:04.209423 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.209927 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.210933 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.211267 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.211342 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:04.213921 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:04.213922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:04.213923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:04.213932 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:04.215724 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.216117 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.216119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.216119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.216179 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.216470 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.216471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.216472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.216607 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:04.216680 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:04.216932 (+ 252us) server_negotiation.cc:299] Negotiation successful
0923 22:08:04.217025 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:08:04.217204 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:04.207154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40375 (local address 127.1.244.194:55323)
0923 22:08:04.207374 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:04.207410 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:04.208584 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:04.208838 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:04.208842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:04.209126 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:04.209320 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:04.209324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.210021 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.210022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:04.210829 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:04.210831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.211362 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.211363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:04.211483 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:04.212128 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:04.212137 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:04.213828 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:04.215816 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:04.215821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:04.215823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:04.216039 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:04.216277 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:04.216278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:04.216278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:04.216387 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:04.216680 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:04.216684 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:04.216812 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 22:08:04.216901 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:08:04.217023 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":106,"threads_started":1}
I20260923 22:08:04.217792 7891 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40375
I20260923 22:08:04.217900 7891 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:04.218076 7891 heartbeater.cc:511] Master 127.1.244.254:40375 requested a full tablet report, sending...
I20260923 22:08:04.218461 7566 ts_manager.cc:194] Registered new tserver with Master: fba608f4bb3c46a384bba8a83465f610 (127.1.244.194:44477)
I20260923 22:08:04.218866 7566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:55323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:08:04.232491 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40375
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:04.312860 7896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:04.313067 7896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:04.313091 7896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:04.314400 7896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:04.314445 7896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:04.314473 7896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:04.316183 7896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:04.316444 7896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:04.316677 7896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:04.321633 7904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:04.321658 7902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:04.321658 7901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:04.321964 7896 server_base.cc:1034] running on GCE node
I20260923 22:08:04.322216 7896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:04.322461 7896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:04.323633 7896 hybrid_clock.cc:648] HybridClock initialized: now 1790201284323606 us; error 34 us; skew 500 ppm
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.326186 7896 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:08:04.326694 7896 webserver.cc:460] Webserver started at http://127.1.244.195:35141/ using document root <none> and password file <none>
I20260923 22:08:04.326943 7896 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:04.327013 7896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:04.327169 7896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:04.328356 7896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "80780522b0914eb2bf572624eec7f8ad"
format_stamp: "Formatted at 2026-09-23 22:08:04 on dist-test-slave-b5jz"
server_key: "c7c4888c1ef9c501fb9c3cc993d25cbf"
server_key_iv: "62f4ecd95055750475ecb7a8ec7ff488"
server_key_version: "encryptionkey@0"
I20260923 22:08:04.328758 7896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "80780522b0914eb2bf572624eec7f8ad"
format_stamp: "Formatted at 2026-09-23 22:08:04 on dist-test-slave-b5jz"
server_key: "c7c4888c1ef9c501fb9c3cc993d25cbf"
server_key_iv: "62f4ecd95055750475ecb7a8ec7ff488"
server_key_version: "encryptionkey@0"
I20260923 22:08:04.330094 7896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:04.330844 7911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:04.331033 7896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:04.331151 7896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "80780522b0914eb2bf572624eec7f8ad"
format_stamp: "Formatted at 2026-09-23 22:08:04 on dist-test-slave-b5jz"
server_key: "c7c4888c1ef9c501fb9c3cc993d25cbf"
server_key_iv: "62f4ecd95055750475ecb7a8ec7ff488"
server_key_version: "encryptionkey@0"
I20260923 22:08:04.331228 7896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:04.377540 7896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:04.379416 7896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:04.379630 7896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:04.379884 7896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:04.380195 7896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:04.380249 7896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:04.380318 7896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:04.380349 7896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:04.386138 7896 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:44789
I20260923 22:08:04.386220 8024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:44789 every 8 connection(s)
I20260923 22:08:04.386555 7896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:04.387501 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 7896
I20260923 22:08:04.387631 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:04.387843 2003 external_mini_cluster.cc:1428] Setting key edeea2a634d3ef2bd1b616e3b9f87695
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:04.396013 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:04.387335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:34823 (local address 127.1.244.254:40375)
0923 22:08:04.387416 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:04.387419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:04.387783 (+ 364us) server_negotiation.cc:407] Connection header received
0923 22:08:04.389055 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:04.389056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:04.389067 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:04.389123 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:04.389755 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.390246 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.391213 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.391406 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.391459 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:04.393630 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:04.393631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:04.393631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:04.393638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:04.394820 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.395180 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.395181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.395181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.395230 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.395442 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.395442 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.395443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.395586 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:04.395656 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:04.395886 (+ 230us) server_negotiation.cc:299] Negotiation successful
0923 22:08:04.395947 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:08:04.396247 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:04.387314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40375 (local address 127.1.244.195:34823)
0923 22:08:04.387651 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:04.387700 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:04.388901 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:04.389179 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:04.389183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:04.389448 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:04.389673 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:04.389677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.390345 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.390346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:04.391139 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:04.391141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.391479 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.391480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:04.391592 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:04.392163 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:04.392179 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:04.393540 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:04.394892 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:04.394897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:04.394899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:04.395118 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:04.395287 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:04.395287 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:04.395288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:04.395390 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:04.395653 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:04.395657 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:04.395772 (+ 115us) client_negotiation.cc:769] Sending connection context
0923 22:08:04.395861 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:08:04.395983 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":101,"threads_started":1}
I20260923 22:08:04.396734 8025 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40375
I20260923 22:08:04.396817 8025 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:04.396963 8025 heartbeater.cc:511] Master 127.1.244.254:40375 requested a full tablet report, sending...
I20260923 22:08:04.397310 7566 ts_manager.cc:194] Registered new tserver with Master: 80780522b0914eb2bf572624eec7f8ad (127.1.244.195:44789)
I20260923 22:08:04.397681 7566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:34823
I20260923 22:08:04.399667 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/test-admin.keytab.
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.415174 2003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260923 22:08:04.416873 8032 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:04.416054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.1.244.193:40959)
0923 22:08:04.416226 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:04.416230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:04.416241 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:08:04.416266 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:04.416268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:04.416283 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:04.416344 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:04.416787 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54690: BlockingRecv error: recv got EOF from 127.0.0.1:54690 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
W20260923 22:08:04.416922 2003 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5cc' of type FILE
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.417940 2003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.418830 2003 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:08:04.426112 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:04.419338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.1.244.193:40959)
0923 22:08:04.419381 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:04.419382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:04.419404 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:08:04.419459 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:04.419460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:04.419469 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:04.419514 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:04.419945 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.420650 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.421365 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.421689 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.421744 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:04.423791 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:04.423799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:04.423799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:04.423806 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:04.425159 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.425473 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.425476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.425476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.425527 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.425685 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.425685 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.425685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.425816 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:04.425916 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:04.425980 (+ 64us) server_negotiation.cc:299] Negotiation successful
0923 22:08:04.426038 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:08:04.428027 8032 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:04.427390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.1.244.193:40959)
0923 22:08:04.427468 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:04.427469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:04.427517 (+ 48us) server_negotiation.cc:407] Connection header received
0923 22:08:04.427625 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:04.427626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:04.427640 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:04.427686 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:04.427962 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54718: BlockingRecv error: recv got EOF from 127.0.0.1:54718 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260923 22:08:04.428064 2003 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5cc' of type FILE
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.429033 2003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.429874 2003 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:08:04.436334 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:04.430376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.1.244.193:40959)
0923 22:08:04.430433 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:04.430434 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:04.430441 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:04.430458 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:04.430459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:04.430468 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:04.430511 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:04.430848 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.431308 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.432005 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.432189 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.432239 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:04.434128 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:04.434129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:04.434130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:04.434136 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:04.435399 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.435799 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.435799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.435800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.435855 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.436003 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.436003 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.436003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.436083 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:04.436144 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:04.436219 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 22:08:04.436275 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260923 22:08:04.437930 8032 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:04.437344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.1.244.193:40959)
0923 22:08:04.437405 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:04.437407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:04.437455 (+ 48us) server_negotiation.cc:407] Connection header received
0923 22:08:04.437527 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:04.437528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:04.437537 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:04.437603 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:04.437885 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54732: BlockingRecv error: recv got EOF from 127.0.0.1:54732 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260923 22:08:04.438037 2003 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201256447402-2003-0/krb5kdc/krb5cc' of type FILE
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.438923 2003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:04.439790 2003 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[7521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:08:04.446159 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:04.440280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.1.244.193:40959)
0923 22:08:04.440351 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:04.440352 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:04.440388 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:08:04.440437 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:04.440438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:04.440445 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:04.440537 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:04.440890 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.441489 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.442109 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:04.442281 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:04.442329 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:04.444227 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:04.444228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:04.444229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:04.444234 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:04.445367 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.445650 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.445650 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.445651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.445697 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:04.445836 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:04.445837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:04.445837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:04.445912 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:04.445973 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:04.446058 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:08:04.446107 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:08:04.446779 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 7628
I20260923 22:08:04.451257 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 7762
I20260923 22:08:04.455459 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 7896
I20260923 22:08:04.459725 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 7534
2026-09-23T22:08:04Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1936 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[8045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[8045](info): set up 2 sockets
Sep 23 22:08:04 dist-test-slave-b5jz krb5kdc[8045](info): commencing operation
krb5kdc: starting...
W20260923 22:08:05.487938 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:08:05 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.254.keytab.
Entry for principal oryx/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.254.keytab.
I20260923 22:08:05.605998 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:34767
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.254.keytab
--principal=oryx/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:34767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:05.685966 8058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:05.686226 8058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:05.686264 8058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:05.687443 8058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:05.687515 8058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:05.687541 8058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:05.687562 8058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:05.687619 8058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:05.689123 8058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:34767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.254.keytab
--principal=oryx/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:34767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:05.689407 8058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:05.689641 8058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:05.694298 8063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:05.694304 8064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:05.694296 8066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:05.694790 8058 server_base.cc:1034] running on GCE node
I20260923 22:08:05.695041 8058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:05.695292 8058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:05.696460 8058 hybrid_clock.cc:648] HybridClock initialized: now 1790201285696447 us; error 46 us; skew 500 ppm
Sep 23 22:08:05 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201285, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:05.698868 8058 init.cc:377] Logged in from keytab as oryx/127.1.244.254@KRBTEST.COM (short username oryx)
I20260923 22:08:05.699344 8058 webserver.cc:460] Webserver started at http://127.1.244.254:44953/ using document root <none> and password file <none>
I20260923 22:08:05.699632 8058 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:05.699700 8058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:05.699857 8058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:05.700910 8058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "bd775ca101fd4cc2ad5778cbff713bcf"
format_stamp: "Formatted at 2026-09-23 22:08:05 on dist-test-slave-b5jz"
server_key: "eb201c390799870a64e6cc1fe42a45bd"
server_key_iv: "2aaf812f5645d734cfa1d34485e53121"
server_key_version: "encryptionkey@0"
I20260923 22:08:05.701277 8058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "bd775ca101fd4cc2ad5778cbff713bcf"
format_stamp: "Formatted at 2026-09-23 22:08:05 on dist-test-slave-b5jz"
server_key: "eb201c390799870a64e6cc1fe42a45bd"
server_key_iv: "2aaf812f5645d734cfa1d34485e53121"
server_key_version: "encryptionkey@0"
I20260923 22:08:05.702577 8058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:05.703281 8073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:05.703454 8058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:05.703572 8058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "bd775ca101fd4cc2ad5778cbff713bcf"
format_stamp: "Formatted at 2026-09-23 22:08:05 on dist-test-slave-b5jz"
server_key: "eb201c390799870a64e6cc1fe42a45bd"
server_key_iv: "2aaf812f5645d734cfa1d34485e53121"
server_key_version: "encryptionkey@0"
I20260923 22:08:05.703662 8058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:05.719412 8058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:05.721398 8058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:05.721566 8058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:05.725719 8058 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:34767
I20260923 22:08:05.725754 8125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:34767 every 8 connection(s)
I20260923 22:08:05.726106 8058 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:05.727082 8126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:05.729348 8126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: Bootstrap starting.
I20260923 22:08:05.729871 8126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:05.730100 8126 log.cc:824] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:05.730726 8126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: No bootstrap required, opened a new log
I20260923 22:08:05.730895 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 8058
I20260923 22:08:05.731002 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:05.731207 2003 external_mini_cluster.cc:1428] Setting key c10a36132db3ad204ecce635ce006f97
I20260923 22:08:05.732841 8126 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } }
I20260923 22:08:05.733019 8126 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:05.733057 8126 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd775ca101fd4cc2ad5778cbff713bcf, State: Initialized, Role: FOLLOWER
I20260923 22:08:05.733227 8126 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } }
I20260923 22:08:05.733321 8126 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:05.733354 8126 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:05.733403 8126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:05.734375 8126 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } }
I20260923 22:08:05.734463 8126 leader_election.cc:302] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd775ca101fd4cc2ad5778cbff713bcf; no voters:
I20260923 22:08:05.734632 8126 leader_election.cc:288] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:05.734654 8131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:05.734759 8131 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 1 LEADER]: Becoming Leader. State: Replica: bd775ca101fd4cc2ad5778cbff713bcf, State: Running, Role: LEADER
I20260923 22:08:05.734918 8126 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:05.734903 8131 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } }
I20260923 22:08:05.736132 8132 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd775ca101fd4cc2ad5778cbff713bcf. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } }
I20260923 22:08:05.736254 8132 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:05.736132 8133 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } }
I20260923 22:08:05.736594 8133 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
W20260923 22:08:05.737090 8145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:08:05.737169 8145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:08:05.737274 8146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:05.737474 8146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:08:05 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:05.743495 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:05.731762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57900 (local address 127.1.244.254:34767)
0923 22:08:05.731942 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:05.731951 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:05.731988 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:08:05.732824 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:05.732830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:05.732878 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:05.732997 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:05.733829 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:05.734921 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:05.736108 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:05.736777 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:05.736894 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:05.739644 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:05.739648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:05.739648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:05.739660 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:05.741590 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:05.741960 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:05.741963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:05.741964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:05.742109 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:05.742305 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:05.742306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:05.742306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:05.742668 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:05.742827 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:05.742911 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 22:08:05.743059 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260923 22:08:05.744194 8146 catalog_manager.cc:1347] Generated new cluster ID: 3f507a557c87407892d6df79cef3c379
I20260923 22:08:05.744246 8146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:05.761193 8146 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:05.761744 8146 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:05.764652 8146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: Generated new TSK 0
I20260923 22:08:05.764871 8146 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
I20260923 22:08:05.797801 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34767
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:05.890034 8152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:05.890250 8152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:05.890275 8152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:05.891628 8152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:05.891671 8152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:05.891719 8152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:05.893229 8152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:05.893473 8152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:05.893716 8152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:05.898675 8157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:05.898667 8160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:05.898664 8158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:05.898942 8152 server_base.cc:1034] running on GCE node
I20260923 22:08:05.899149 8152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:05.899408 8152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:05.900552 8152 hybrid_clock.cc:648] HybridClock initialized: now 1790201285900535 us; error 31 us; skew 500 ppm
Sep 23 22:08:05 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201285, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:05.903079 8152 init.cc:377] Logged in from keytab as oryx/127.1.244.193@KRBTEST.COM (short username oryx)
I20260923 22:08:05.903527 8152 webserver.cc:460] Webserver started at http://127.1.244.193:35965/ using document root <none> and password file <none>
I20260923 22:08:05.903777 8152 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:05.903821 8152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:05.903914 8152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:05.904932 8152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "0ec38f76df844757bbcfec9a04b56e98"
format_stamp: "Formatted at 2026-09-23 22:08:05 on dist-test-slave-b5jz"
server_key: "94553e2e73acabe8eeafce05a9c6302b"
server_key_iv: "bff75873e6dddc936e23f1769a0d7c4d"
server_key_version: "encryptionkey@0"
I20260923 22:08:05.905232 8152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "0ec38f76df844757bbcfec9a04b56e98"
format_stamp: "Formatted at 2026-09-23 22:08:05 on dist-test-slave-b5jz"
server_key: "94553e2e73acabe8eeafce05a9c6302b"
server_key_iv: "bff75873e6dddc936e23f1769a0d7c4d"
server_key_version: "encryptionkey@0"
I20260923 22:08:05.906546 8152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:05.907267 8167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:05.907461 8152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:05.907578 8152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "0ec38f76df844757bbcfec9a04b56e98"
format_stamp: "Formatted at 2026-09-23 22:08:05 on dist-test-slave-b5jz"
server_key: "94553e2e73acabe8eeafce05a9c6302b"
server_key_iv: "bff75873e6dddc936e23f1769a0d7c4d"
server_key_version: "encryptionkey@0"
I20260923 22:08:05.907651 8152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:05.920238 8152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:05.922580 8152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:05.922696 8152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:05.922879 8152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:05.923211 8152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:05.923249 8152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:05.923272 8152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:05.923285 8152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:05.929040 8152 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:33885
I20260923 22:08:05.929123 8280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:33885 every 8 connection(s)
I20260923 22:08:05.929464 8152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:05.933291 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 8152
I20260923 22:08:05.933389 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:05.933597 2003 external_mini_cluster.cc:1428] Setting key be7f1404598681c2c485e42f83ec1a01
Sep 23 22:08:05 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201285, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:05.939774 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:05.930287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:49297 (local address 127.1.244.254:34767)
0923 22:08:05.930338 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:05.930340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:05.930577 (+ 237us) server_negotiation.cc:407] Connection header received
0923 22:08:05.931119 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:05.931121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:05.931139 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:05.931198 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:05.932180 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:05.932687 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:05.933843 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:05.934060 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:05.934148 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:05.936802 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:05.936804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:05.936804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:05.936812 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:05.938307 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:05.938743 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:05.938744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:05.938745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:05.938811 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:05.939074 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:05.939075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:05.939076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:05.939197 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:05.939272 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:05.939584 (+ 312us) server_negotiation.cc:299] Negotiation successful
0923 22:08:05.939682 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:08:05.939951 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:05.930254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.193:49297)
0923 22:08:05.930448 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:05.930484 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:05.931004 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:05.931269 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:05.931273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:05.931878 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:05.932072 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:05.932077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:05.932789 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:05.932790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:05.933745 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:05.933747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:05.934157 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:05.934157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:05.934294 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:05.934838 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:05.934847 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:05.936685 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:05.938393 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:05.938399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:05.938400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:05.938653 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:05.938897 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:05.938897 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:05.938898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:05.939002 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:05.939274 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:05.939280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:05.939417 (+ 137us) client_negotiation.cc:769] Sending connection context
0923 22:08:05.939523 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:08:05.939703 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260923 22:08:05.940514 8281 heartbeater.cc:347] Connected to a master server at 127.1.244.254:34767
I20260923 22:08:05.940598 8281 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:05.940790 8281 heartbeater.cc:511] Master 127.1.244.254:34767 requested a full tablet report, sending...
I20260923 22:08:05.941422 8089 ts_manager.cc:194] Registered new tserver with Master: 0ec38f76df844757bbcfec9a04b56e98 (127.1.244.193:33885)
I20260923 22:08:05.942373 8089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:49297
WARNING: no policy specified for oryx/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
I20260923 22:08:05.959260 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34767
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:06.041957 8286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:06.042248 8286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:06.042282 8286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:06.043931 8286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:06.043987 8286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:06.044018 8286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:06.045455 8286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:06.045651 8286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:06.045840 8286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.050776 8291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.050786 8292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.050953 8286 server_base.cc:1034] running on GCE node
W20260923 22:08:06.050787 8294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.051290 8286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.051632 8286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.052809 8286 hybrid_clock.cc:648] HybridClock initialized: now 1790201286052792 us; error 37 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.055369 8286 init.cc:377] Logged in from keytab as oryx/127.1.244.194@KRBTEST.COM (short username oryx)
I20260923 22:08:06.055889 8286 webserver.cc:460] Webserver started at http://127.1.244.194:44903/ using document root <none> and password file <none>
I20260923 22:08:06.056123 8286 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.056192 8286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.056352 8286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:06.057446 8286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "5567facef3cf4b4f845f9af30f43580b"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "a211a4b0f52aaacb27822a9da8c73be9"
server_key_iv: "7981dd5d2aaff7e2d76a75b4361ed6d6"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.057819 8286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "5567facef3cf4b4f845f9af30f43580b"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "a211a4b0f52aaacb27822a9da8c73be9"
server_key_iv: "7981dd5d2aaff7e2d76a75b4361ed6d6"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.059134 8286 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:06.059917 8301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.060078 8286 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.060189 8286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "5567facef3cf4b4f845f9af30f43580b"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "a211a4b0f52aaacb27822a9da8c73be9"
server_key_iv: "7981dd5d2aaff7e2d76a75b4361ed6d6"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.060264 8286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.073791 8286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.076297 8286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:06.076476 8286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.076723 8286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:06.077045 8286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:06.077102 8286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.077176 8286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:06.077208 8286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.083063 8286 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:39369
I20260923 22:08:06.083112 8414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:39369 every 8 connection(s)
I20260923 22:08:06.083434 8286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:06.084394 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 8286
I20260923 22:08:06.084504 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:06.084703 2003 external_mini_cluster.cc:1428] Setting key 883b8e9adf0080e10da800b782ed11c3
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.244.195@KRBTEST.COM; defaulting to no policy
I20260923 22:08:06.093751 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.084153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:42855 (local address 127.1.244.254:34767)
0923 22:08:06.084224 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.084227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.084492 (+ 265us) server_negotiation.cc:407] Connection header received
0923 22:08:06.085163 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.085164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.085176 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.085233 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.086246 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.086752 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.087787 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.088100 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.088172 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.090651 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.090652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.090653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.090662 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.092325 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.092745 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.092746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.092747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.092817 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.093085 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.093086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.093086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.093206 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.093284 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.093562 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.093661 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:08:06.093918 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.084131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.194:42855)
0923 22:08:06.084334 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.084382 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.085056 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.085315 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.085319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.085951 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.086138 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.086144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.086850 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.086851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.087693 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.087696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.088186 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.088187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.088298 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.088852 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.088861 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.090550 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.092430 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.092438 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.092440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.092666 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.092886 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.092887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.092887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.093012 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.093275 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.093281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.093424 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.093529 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.093686 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":134,"threads_started":1}
I20260923 22:08:06.094416 8415 heartbeater.cc:347] Connected to a master server at 127.1.244.254:34767
I20260923 22:08:06.094532 8415 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:06.094744 8415 heartbeater.cc:511] Master 127.1.244.254:34767 requested a full tablet report, sending...
I20260923 22:08:06.095208 8089 ts_manager.cc:194] Registered new tserver with Master: 5567facef3cf4b4f845f9af30f43580b (127.1.244.194:39369)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.195@KRBTEST.COM" created.
I20260923 22:08:06.095680 8089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:42855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
I20260923 22:08:06.110548 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34767
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:06.191080 8420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:06.191308 8420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:06.191331 8420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:06.192515 8420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:06.192560 8420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:06.192587 8420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:06.194047 8420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:06.194291 8420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:06.194541 8420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.199438 8425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.199496 8426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.199421 8428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.199648 8420 server_base.cc:1034] running on GCE node
I20260923 22:08:06.199839 8420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.200095 8420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.201233 8420 hybrid_clock.cc:648] HybridClock initialized: now 1790201286201225 us; error 29 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.203758 8420 init.cc:377] Logged in from keytab as oryx/127.1.244.195@KRBTEST.COM (short username oryx)
I20260923 22:08:06.204234 8420 webserver.cc:460] Webserver started at http://127.1.244.195:41857/ using document root <none> and password file <none>
I20260923 22:08:06.204418 8420 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.204458 8420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.204545 8420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:06.205555 8420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "e58c1112264e4386a4cce33487fe2dc3"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "93f2412032ce6038a1173d61caac52e5"
server_key_iv: "878527868e857696d82bd39eed319a85"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.205854 8420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "e58c1112264e4386a4cce33487fe2dc3"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "93f2412032ce6038a1173d61caac52e5"
server_key_iv: "878527868e857696d82bd39eed319a85"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.207198 8420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:06.208019 8435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.208177 8420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.208277 8420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "e58c1112264e4386a4cce33487fe2dc3"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "93f2412032ce6038a1173d61caac52e5"
server_key_iv: "878527868e857696d82bd39eed319a85"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.208355 8420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.217047 8420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.218730 8420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:06.218894 8420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.219105 8420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:06.219400 8420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:06.219455 8420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.219497 8420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:06.219524 8420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.225404 8420 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:42541
I20260923 22:08:06.225452 8548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:42541 every 8 connection(s)
I20260923 22:08:06.225745 8420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:06.234050 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.226396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:36751 (local address 127.1.244.254:34767)
0923 22:08:06.226438 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.226440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.226621 (+ 181us) server_negotiation.cc:407] Connection header received
0923 22:08:06.227084 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.227085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.227096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.227146 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.228018 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.228506 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.229423 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.229598 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.229646 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.231587 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.231588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.231589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.231596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.232978 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.233310 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.233310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.233311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.233360 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.233557 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.233558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.233558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.233655 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.233702 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.233928 (+ 226us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.233987 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:06.234198 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.226391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:36751)
0923 22:08:06.226524 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.226555 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.226997 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.227197 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.227200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.227759 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.227944 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.227948 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.228583 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.228584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.229353 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.229354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.229666 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.229666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.229759 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.230177 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.230185 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.231471 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.233045 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.233049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.233051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.233246 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.233416 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.233416 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.233417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.233504 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.233707 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.233710 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.233813 (+ 103us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.233899 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.234010 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260923 22:08:06.234695 8549 heartbeater.cc:347] Connected to a master server at 127.1.244.254:34767
I20260923 22:08:06.234773 8549 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:06.234962 8549 heartbeater.cc:511] Master 127.1.244.254:34767 requested a full tablet report, sending...
I20260923 22:08:06.235350 8089 ts_manager.cc:194] Registered new tserver with Master: e58c1112264e4386a4cce33487fe2dc3 (127.1.244.195:42541)
I20260923 22:08:06.235697 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 8420
I20260923 22:08:06.235800 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:06.235903 8089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:36751
I20260923 22:08:06.236012 2003 external_mini_cluster.cc:1428] Setting key b9d86b0a18e44a128b3d174be08678cf
I20260923 22:08:06.236845 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.244.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.253.keytab.
Entry for principal oryx/127.1.244.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.253.keytab.
I20260923 22:08:06.262095 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:41969
--webserver_interface=127.1.244.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.253.keytab
--principal=oryx/127.1.244.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:34767,127.1.244.253:41969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:06.342875 8554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:06.343079 8554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:06.343101 8554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:06.344177 8554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:06.344213 8554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:06.344226 8554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:06.344239 8554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:06.344251 8554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:06.345654 8554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:34767,127.1.244.253:41969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.253.keytab
--principal=oryx/127.1.244.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.244.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
W20260923 22:08:06.345755 8554 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.244.254:34767,127.1.244.253:41969'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260923 22:08:06.345870 8554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:06.346029 8554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.350605 8560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.350600 8559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.350608 8562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.350955 8554 server_base.cc:1034] running on GCE node
I20260923 22:08:06.351192 8554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.351503 8554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.352695 8554 hybrid_clock.cc:648] HybridClock initialized: now 1790201286352683 us; error 28 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.355034 8554 init.cc:377] Logged in from keytab as oryx/127.1.244.253@KRBTEST.COM (short username oryx)
I20260923 22:08:06.355412 8554 webserver.cc:460] Webserver started at http://127.1.244.253:39791/ using document root <none> and password file <none>
I20260923 22:08:06.355671 8554 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.355717 8554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.355859 8554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:06.356921 8554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data/instance:
uuid: "ae670858be364ed6bc4c88d32a1eced2"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "1cff59deb3db398b44ffc6b7a387341d"
server_key_iv: "c88c24b155d6faa55b07a2d9a31ea0bc"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.357296 8554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal/instance:
uuid: "ae670858be364ed6bc4c88d32a1eced2"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "1cff59deb3db398b44ffc6b7a387341d"
server_key_iv: "c88c24b155d6faa55b07a2d9a31ea0bc"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.358599 8554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:06.359354 8569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.359529 8554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:06.359673 8554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal
uuid: "ae670858be364ed6bc4c88d32a1eced2"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "1cff59deb3db398b44ffc6b7a387341d"
server_key_iv: "c88c24b155d6faa55b07a2d9a31ea0bc"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.359746 8554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.374586 8554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.376374 8554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:06.376533 8554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.380801 8554 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:41969
I20260923 22:08:06.380832 8621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:41969 every 8 connection(s)
I20260923 22:08:06.381193 8554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
I20260923 22:08:06.382145 8622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:06.383674 8622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:08:06.387099 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 8554
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.253@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260923 22:08:06.393062 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.384404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.1.244.254:34767)
0923 22:08:06.384471 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.384473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.384678 (+ 205us) server_negotiation.cc:407] Connection header received
0923 22:08:06.385190 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.385192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.385205 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.385255 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.385863 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.386371 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.387435 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.387742 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.387840 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.390037 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.390038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.390039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.390046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.391499 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.392025 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.392026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.392026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.392093 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.392334 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.392335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.392336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.392530 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.392607 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.392885 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.392970 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:08:06.393253 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.384378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:57906)
0923 22:08:06.384561 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.384596 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.385067 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.385326 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.385329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.385574 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.385766 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.385770 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.386482 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.386483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.387332 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.387335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.387857 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.387858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.388030 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.388590 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.388604 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.389945 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.391626 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.391632 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.391634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.391950 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.392161 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.392161 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.392162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.392264 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.392604 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.392610 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.392741 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.392827 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.392936 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260923 22:08:06.394037 8622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:08:06.396186 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.387618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.1.244.253:41969)
0923 22:08:06.387759 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.387763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.387868 (+ 105us) server_negotiation.cc:407] Connection header received
0923 22:08:06.387910 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.387914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.387934 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.388006 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.388571 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.389225 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.390070 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.390348 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.390429 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.393021 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.393026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.393026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.393032 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.394929 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.395313 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.395315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.395315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.395367 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.395593 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.395594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.395594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.395745 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.395922 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.396012 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.396087 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.253@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260923 22:08:06.401119 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.394288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.0.0.1:37478)
0923 22:08:06.394339 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.394350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.394393 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.394624 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.394625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.394880 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.394939 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.394940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.395757 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.395758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.396509 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.396511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.396841 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.396842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.396907 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.397328 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.397332 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.398385 (+ 1053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.400274 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.400275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.400276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.400443 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.400620 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.400621 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.400621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.400687 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.400882 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.400883 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.400915 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.400965 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.401041 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:08:06.401203 8627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.394376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.1.244.253:41969)
0923 22:08:06.394500 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.394502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.394508 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:08:06.394530 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.394530 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.394537 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.394587 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.395011 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.395692 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.396578 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.396776 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.396826 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.398456 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.398457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.398458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.398463 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.400184 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.400512 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.400513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.400513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.400562 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.400739 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.400740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.400740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.400831 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.400880 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.400981 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.401084 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
I20260923 22:08:06.402447 8622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: Bootstrap starting.
I20260923 22:08:06.403085 8622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:06.403365 8622 log.cc:824] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:06.404243 8622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: No bootstrap required, opened a new log
I20260923 22:08:06.405647 8622 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } }
I20260923 22:08:06.405769 8622 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:06.405815 8622 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae670858be364ed6bc4c88d32a1eced2, State: Initialized, Role: FOLLOWER
I20260923 22:08:06.405997 8622 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } }
I20260923 22:08:06.406263 8629 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } } }
I20260923 22:08:06.406373 8629 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:06.406306 8622 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: configured and running, proceeding with master startup.
W20260923 22:08:06.408087 8640 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:08:06.408177 8640 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:08:06.409597 8554 master_runner.cc:428] Detected that this master ae670858be364ed6bc4c88d32a1eced2 is joining an existing cluster
I20260923 22:08:06.409655 8554 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.244.253:41969
I20260923 22:08:06.417531 8627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.411005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.1.244.253:41969)
0923 22:08:06.411040 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.411042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.411248 (+ 206us) server_negotiation.cc:407] Connection header received
0923 22:08:06.411325 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.411326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.411333 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.411360 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.411893 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.412535 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.413280 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.413506 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.413588 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.414635 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.414636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.414637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.414643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.416278 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.416630 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.416631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.416632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.416680 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.416916 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.416917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.416918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.417054 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.417126 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.417275 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.417431 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":19}
I20260923 22:08:06.417723 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.411075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.0.0.1:37488)
0923 22:08:06.411198 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.411207 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.411267 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.411421 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.411422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.411637 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.411693 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.411694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.412619 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.412620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.413184 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.413185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.413604 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.413605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.413661 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.414136 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.414141 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.414552 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.416378 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.416379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.416380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.416546 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.416766 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.416766 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.416766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.416841 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.417114 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.417115 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.417176 (+ 61us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.417254 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.417363 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":3,"thread_start_us":75,"threads_started":1}
I20260923 22:08:06.418332 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.410721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:57908)
0923 22:08:06.410922 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.410932 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.411042 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.411194 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.411196 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.411427 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.411519 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.411521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.412677 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.412678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.413552 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.413554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.414082 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.414083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.414143 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.414544 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.414549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.414921 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.416750 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.416751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.416751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.416987 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.417187 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.417188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.417188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.417247 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.418014 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.418015 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.418047 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.418139 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.418232 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":113,"threads_started":1}
I20260923 22:08:06.418349 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.410793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.1.244.254:34767)
0923 22:08:06.410864 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.410866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.411015 (+ 149us) server_negotiation.cc:407] Connection header received
0923 22:08:06.411104 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.411105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.411118 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.411176 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.411660 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.412577 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.413653 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.413963 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.414005 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.415025 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.415027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.415027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.415033 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.416670 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.417059 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.417060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.417061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.417120 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.417340 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.417341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.417342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.417483 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.417557 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.418175 (+ 618us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.418259 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201285, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260923 22:08:06.430882 8627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.420763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37492 (local address 127.1.244.253:41969)
0923 22:08:06.420797 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.420799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.423776 (+ 2977us) server_negotiation.cc:407] Connection header received
0923 22:08:06.423879 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.423880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.423888 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.423935 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.424456 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.424885 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.425995 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.426159 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.426209 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.428350 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.428351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.428351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.428358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.429677 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.430100 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.430102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.430102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.430157 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.430351 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.430352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.430352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.430459 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.430527 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.430720 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.430793 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:08:06.430902 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.420722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.0.0.1:37492)
0923 22:08:06.423662 (+ 2940us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.423711 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.423791 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.423962 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.423967 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.424315 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.424379 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.424381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.424957 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.424958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.425918 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.425921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.426231 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.426232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.426326 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.426735 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.426740 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.428259 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.429794 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.429801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.429803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.430012 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.430220 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.430221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.430222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.430288 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.430534 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.430541 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.430590 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.430681 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.430796 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2908,"thread_start_us":57,"threads_started":1}
I20260923 22:08:06.431408 8089 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.244.253:41969: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2"
member_type: NON_VOTER
last_known_addr {
host: "127.1.244.253"
port: 41969
}
attrs {
promote: true
}
}
dest_uuid: "bd775ca101fd4cc2ad5778cbff713bcf"
cas_config_opid_index: -1
I20260923 22:08:06.431769 8089 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } }
I20260923 22:08:06.432269 8650 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ae670858be364ed6bc4c88d32a1eced2 (127.1.244.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } } }
I20260923 22:08:06.432576 8596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:06.433369 8650 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.244.253:41969
I20260923 22:08:06.433470 8596 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Refusing update from remote peer bd775ca101fd4cc2ad5778cbff713bcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260923 22:08:06.433457 8650 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } } }
I20260923 22:08:06.433521 8650 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:06.433835 8554 master.cc:537] Master@127.1.244.253:41969 shutting down...
I20260923 22:08:06.433835 8650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:08:06.434721 8076 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260923 22:08:06.435204 8076 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf -> Peer ae670858be364ed6bc4c88d32a1eced2 (127.1.244.253:41969): Couldn't send request to peer ae670858be364ed6bc4c88d32a1eced2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:08:06.436502 8554 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:06.436587 8554 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:06.436637 8554 tablet_replica.cc:330] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: stopping tablet replica
I20260923 22:08:06.449378 8554 master.cc:552] Master@127.1.244.253:41969 shutdown complete.
I20260923 22:08:06.449450 8554 master_runner.cc:305] Clearing existing system tablet
I20260923 22:08:06.449575 8554 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.449637 8554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.450237 8554 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260923 22:08:06.450709 8654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.450846 8554 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.450915 8554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.450942 8554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal
uuid: "ae670858be364ed6bc4c88d32a1eced2"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "1cff59deb3db398b44ffc6b7a387341d"
server_key_iv: "c88c24b155d6faa55b07a2d9a31ea0bc"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.451787 8554 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:06.453333 8554 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 22:08:06.453406 8554 log.cc:1196] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260923 22:08:06.453608 8554 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: Deleting consensus metadata
I20260923 22:08:06.453733 8554 master_runner.cc:315] Copying system tablet from 127.1.244.254:34767
I20260923 22:08:06.454401 8554 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: tablet copy: Beginning tablet copy session from remote peer at address 127.1.244.254:34767
I20260923 22:08:06.460932 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.454801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.1.244.254:34767)
0923 22:08:06.454862 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.454864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.455020 (+ 156us) server_negotiation.cc:407] Connection header received
0923 22:08:06.455086 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.455087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.455096 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.455141 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.455494 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.456032 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.457115 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.457307 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.457358 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.458357 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.458358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.458358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.458371 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.459849 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.460232 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.460232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.460233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.460284 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.460501 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.460501 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.460501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.460603 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.460654 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.460809 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.460870 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:08:06.460994 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.454719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:57922)
0923 22:08:06.454932 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.454944 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.455020 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.455145 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.455145 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.455349 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.455419 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.455421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.456117 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.456118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.457031 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.457034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.457376 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.457377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.457444 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.457875 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.457880 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.458265 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.459927 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.459928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.459928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.460151 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.460350 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.460351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.460351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.460421 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.460666 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.460667 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.460713 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.460784 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.460887 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":68,"threads_started":1}
I20260923 22:08:06.461599 8110 tablet_copy_service.cc:140] P bd775ca101fd4cc2ad5778cbff713bcf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ae670858be364ed6bc4c88d32a1eced2 ({username='oryx', principal='oryx/127.1.244.253@KRBTEST.COM'} at 127.0.0.1:57922)
I20260923 22:08:06.461681 8110 tablet_copy_service.cc:161] P bd775ca101fd4cc2ad5778cbff713bcf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ae670858be364ed6bc4c88d32a1eced2 at {username='oryx', principal='oryx/127.1.244.253@KRBTEST.COM'} at 127.0.0.1:57922: session id = ae670858be364ed6bc4c88d32a1eced2-00000000000000000000000000000000
I20260923 22:08:06.462374 8110 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 22:08:06.463033 8554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 22:08:06.464285 8554 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: tablet copy: Starting download of 0 data blocks...
I20260923 22:08:06.464447 8554 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: tablet copy: Starting download of 1 WAL segments...
I20260923 22:08:06.465288 8554 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 22:08:06.466187 8110 tablet_copy_service.cc:342] P bd775ca101fd4cc2ad5778cbff713bcf: Request end of tablet copy session ae670858be364ed6bc4c88d32a1eced2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.244.253@KRBTEST.COM'} at 127.0.0.1:57922
I20260923 22:08:06.466249 8110 tablet_copy_service.cc:433] P bd775ca101fd4cc2ad5778cbff713bcf: ending tablet copy session ae670858be364ed6bc4c88d32a1eced2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ae670858be364ed6bc4c88d32a1eced2
I20260923 22:08:06.468815 8554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.469828 8661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.469893 8664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.469907 8662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.471740 8554 server_base.cc:1034] running on GCE node
I20260923 22:08:06.471879 8554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.472054 8554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.473181 8554 hybrid_clock.cc:648] HybridClock initialized: now 1790201286473177 us; error 28 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.474442 8554 init.cc:377] Logged in from keytab as oryx/127.1.244.253@KRBTEST.COM (short username oryx)
I20260923 22:08:06.474733 8554 webserver.cc:460] Webserver started at http://127.1.244.253:41775/ using document root <none> and password file <none>
I20260923 22:08:06.474872 8554 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.474913 8554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.475538 8554 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:06.476089 8671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.476208 8554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:06.476254 8554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal
uuid: "ae670858be364ed6bc4c88d32a1eced2"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "1cff59deb3db398b44ffc6b7a387341d"
server_key_iv: "c88c24b155d6faa55b07a2d9a31ea0bc"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.476310 8554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.504029 8554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.505743 8554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.508955 8554 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:41969
I20260923 22:08:06.508971 8723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:41969 every 8 connection(s)
I20260923 22:08:06.509398 8554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
I20260923 22:08:06.510335 8724 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:08:06.510856 8724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: Bootstrap starting.
I20260923 22:08:06.513674 8724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:08:06.513886 8724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: Bootstrap complete.
I20260923 22:08:06.514066 8724 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } }
I20260923 22:08:06.514148 8724 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ae670858be364ed6bc4c88d32a1eced2, State: Initialized, Role: LEARNER
I20260923 22:08:06.514181 8724 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } }
I20260923 22:08:06.514341 8724 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:06.514330 8727 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } } }
I20260923 22:08:06.514442 8727 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:08:06.515816 8738 catalog_manager.cc:1259] Loaded cluster ID: 3f507a557c87407892d6df79cef3c379
I20260923 22:08:06.515864 8738 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: loading cluster ID for follower catalog manager: success
I20260923 22:08:06.516861 8738 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: acquiring CA information for follower catalog manager: success
I20260923 22:08:06.517084 8738 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:08:06.554704 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.548673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.1.244.253:41969)
0923 22:08:06.548800 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.548802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.548811 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:06.548835 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.548836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.548844 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.548924 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.549602 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.550232 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.551182 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.551359 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.551409 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.552539 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.552541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.552541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.552547 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.553765 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.554068 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.554068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.554069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.554120 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.554278 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.554278 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.554278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.554373 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.554442 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.554528 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.554588 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for oryx/127.1.244.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.252.keytab.
Entry for principal oryx/127.1.244.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.252.keytab.
I20260923 22:08:06.578181 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:38891
--webserver_interface=127.1.244.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.252.keytab
--principal=oryx/127.1.244.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:34767,127.1.244.253:41969,127.1.244.252:38891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:06.660228 8742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:06.660440 8742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:06.660462 8742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:06.661561 8742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:06.661593 8742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:06.661607 8742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:06.661620 8742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:06.661631 8742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:06.663234 8742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:34767,127.1.244.253:41969,127.1.244.252:38891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.252.keytab
--principal=oryx/127.1.244.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:38891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.244.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:06.663430 8742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:06.663671 8742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.668906 8748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.668850 8750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.668872 8747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.669072 8742 server_base.cc:1034] running on GCE node
I20260923 22:08:06.669374 8742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.669672 8742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.670837 8742 hybrid_clock.cc:648] HybridClock initialized: now 1790201286670828 us; error 38 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.673296 8742 init.cc:377] Logged in from keytab as oryx/127.1.244.252@KRBTEST.COM (short username oryx)
I20260923 22:08:06.673722 8742 webserver.cc:460] Webserver started at http://127.1.244.252:45603/ using document root <none> and password file <none>
I20260923 22:08:06.673969 8742 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.674038 8742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.674185 8742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:06.675230 8742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data/instance:
uuid: "02c291ffca5f4e84a28d99b3947d4c07"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "498c5ac7350005e1a15b5cd995cff08d"
server_key_iv: "c25ae0692276cab48ab722bdbce4548b"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.675642 8742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal/instance:
uuid: "02c291ffca5f4e84a28d99b3947d4c07"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "498c5ac7350005e1a15b5cd995cff08d"
server_key_iv: "c25ae0692276cab48ab722bdbce4548b"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.676983 8742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:06.677788 8757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.677958 8742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:06.678063 8742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal
uuid: "02c291ffca5f4e84a28d99b3947d4c07"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "498c5ac7350005e1a15b5cd995cff08d"
server_key_iv: "c25ae0692276cab48ab722bdbce4548b"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.678153 8742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.705104 8742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.707146 8742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:06.707319 8742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.712194 8742 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:38891
I20260923 22:08:06.712237 8809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:38891 every 8 connection(s)
I20260923 22:08:06.712749 8742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
I20260923 22:08:06.713049 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 8742
I20260923 22:08:06.713842 8810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:06.716035 8810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:06.725111 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.713735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.1.244.252:38891)
0923 22:08:06.713926 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.713936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.713970 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:08:06.714494 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.714501 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.714557 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.714715 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.715345 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.716471 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.717540 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.718131 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.718246 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.720897 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.720904 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.720904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.720912 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.722860 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.723283 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.723287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.723287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.723393 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.723663 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.723664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.723664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.724020 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.724179 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.724679 (+ 500us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.724833 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260923 22:08:06.725283 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.716845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.1.244.254:34767)
0923 22:08:06.716895 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.716897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.716950 (+ 53us) server_negotiation.cc:407] Connection header received
0923 22:08:06.717040 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.717041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.717049 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.717101 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.717682 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.718433 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.719238 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.719455 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.719562 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.721883 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.721885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.721885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.721893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.723822 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.724340 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.724341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.724341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.724405 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.724629 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.724630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.724631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.724773 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.724819 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.725083 (+ 264us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.725193 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:08:06.725299 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.716670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:57928)
0923 22:08:06.716847 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.716891 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.716975 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.717138 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.717154 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.717511 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.717593 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.717595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.718534 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.718535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.719143 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.719144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.719718 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.719719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.719802 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.720294 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.720303 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.721736 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.723933 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.723941 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.723958 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.724230 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.724484 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.724485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.724485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.724552 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.724867 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.724874 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.724928 (+ 54us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.725024 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.725144 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
I20260923 22:08:06.729595 8810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260923 22:08:06.736725 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.729869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.0.0.1:37538)
0923 22:08:06.729928 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.729940 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.729985 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.730111 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.730112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.730281 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.730342 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.730344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.731171 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.731172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.731953 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.731955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.732320 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.732321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.732373 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.732717 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.732721 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.733902 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.735646 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.735646 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.735647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.735866 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.736091 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.736092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.736092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.736162 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.736407 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.736408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.736469 (+ 61us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.736531 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.736628 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 22:08:06.736732 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.729957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.1.244.253:41969)
0923 22:08:06.730001 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.730003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.730011 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:06.730038 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.730039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.730051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.730107 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.730426 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.731084 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.732031 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.732250 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.732310 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.734008 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.734009 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.734009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.734015 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.735580 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.735950 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.735951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.735951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.736015 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.736225 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.736226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.736227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.736331 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.736402 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.736556 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.736634 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:06.737124 8810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
I20260923 22:08:06.742988 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.737370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.0.0.1:60980)
0923 22:08:06.737403 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.737414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.737464 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.737580 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.737581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.737705 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.737727 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.737728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.738325 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.738326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.738792 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.738794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.739165 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.739166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.739212 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.739535 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.739539 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.740683 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.742147 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.742148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.742148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.742310 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.742478 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.742478 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.742478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.742525 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.742758 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.742759 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.742790 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.742837 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.742910 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 22:08:06.743043 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.737397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.1.244.252:38891)
0923 22:08:06.737425 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.737427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.737450 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:08:06.737517 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.737517 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.737526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.737568 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.737798 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.738251 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.738883 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.739080 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.739167 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.740755 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.740757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.740757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.740762 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.742072 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.742375 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.742375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.742376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.742421 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.742588 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.742589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.742590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.742703 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.742754 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.742860 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.742953 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":14}
I20260923 22:08:06.744427 8810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Bootstrap starting.
I20260923 22:08:06.745249 8810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:06.745523 8810 log.cc:824] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:06.746228 8810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: No bootstrap required, opened a new log
I20260923 22:08:06.747705 8810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } }
I20260923 22:08:06.747841 8810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:06.747903 8810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c291ffca5f4e84a28d99b3947d4c07, State: Initialized, Role: FOLLOWER
I20260923 22:08:06.748045 8810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } }
I20260923 22:08:06.748291 8815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } } }
I20260923 22:08:06.748358 8810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:06.748374 8815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:08:06.749979 8826 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:08:06.750051 8826 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:08:06.752316 8742 master_runner.cc:428] Detected that this master 02c291ffca5f4e84a28d99b3947d4c07 is joining an existing cluster
I20260923 22:08:06.752363 8742 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.244.252:38891
I20260923 22:08:06.760890 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.753552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.1.244.252:38891)
0923 22:08:06.753697 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.753698 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.753846 (+ 148us) server_negotiation.cc:407] Connection header received
0923 22:08:06.753904 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.753905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.753913 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.753962 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.754374 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.755086 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.756201 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.756438 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.756481 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.758444 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.758445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.758445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.758451 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.759844 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.760229 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.760230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.760230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.760276 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.760466 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.760466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.760467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.760555 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.760602 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.760763 (+ 161us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.760826 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":126}
I20260923 22:08:06.761008 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.753367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.0.0.1:60990)
0923 22:08:06.753795 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.753805 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.753857 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.753995 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.753996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.754237 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.754294 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.754296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.755185 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.755186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.756095 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.756098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.756557 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.756558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.756620 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.757047 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.757052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.757903 (+ 851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.759921 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.759923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.759923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.760152 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.760337 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.760338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.760338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.760405 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.760616 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.760617 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.760660 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.760750 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.760891 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":86,"spinlock_wait_cycles":9984,"thread_start_us":114,"threads_started":1}
I20260923 22:08:06.776589 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 8152
I20260923 22:08:06.780849 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.753734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37546 (local address 127.1.244.253:41969)
0923 22:08:06.753786 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.753788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.753992 (+ 204us) server_negotiation.cc:407] Connection header received
0923 22:08:06.754082 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.754083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.754093 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.754123 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.754520 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.755207 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.756253 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.756547 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.756584 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.777378 (+ 20794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.777380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.777380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.777389 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.778951 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.779348 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.779349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.779349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.779667 (+ 318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.780186 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.780187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.780188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.780366 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.780418 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.780641 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.780727 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:08:06.781203 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.753562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.0.0.1:37546)
0923 22:08:06.753942 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.753951 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.754013 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.754187 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.754188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.754371 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.754437 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.754438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.755290 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.755290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.756155 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.756157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.756662 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.756663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.756731 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.776513 (+ 19782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.776519 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.777254 (+ 735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.779049 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.779051 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.779051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.779264 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.779791 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.779792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.779792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.779867 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.780481 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.780481 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.780529 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.780975 (+ 446us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.781104 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":30,"thread_start_us":60,"threads_started":1}
I20260923 22:08:06.782771 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:33885
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:34767
--webserver_port=35965
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.244.254:34767,127.1.244.253:41969
--tserver_master_addrs=127.1.244.254:34767,127.1.244.253:41969,127.1.244.252:38891 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:08:06.783349 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.753398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57938 (local address 127.1.244.254:34767)
0923 22:08:06.753453 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.753457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.753638 (+ 181us) server_negotiation.cc:407] Connection header received
0923 22:08:06.753710 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.753711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.753724 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.753782 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.754293 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.755086 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.756254 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.756543 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.756584 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.779154 (+ 22570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.779156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.779156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.779165 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.780832 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.782410 (+ 1578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.782411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.782412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.782472 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.782716 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.782717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.782717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.782870 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.782940 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.783132 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.783220 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:08:06.783767 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.753366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:57938)
0923 22:08:06.753546 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.753558 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.753636 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.753871 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.753872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.754123 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.754207 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.754209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.755179 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.755180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.756155 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.756157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.756661 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.756662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.756731 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.776695 (+ 19964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.776700 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.778424 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.781233 (+ 2809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.781234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.781234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.782321 (+ 1087us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.782559 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.782560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.782560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.782635 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.782969 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.782970 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.783013 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.783473 (+ 460us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.783609 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":4,"thread_start_us":78,"threads_started":1}
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201285, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
I20260923 22:08:06.792475 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.786071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.1.244.252:38891)
0923 22:08:06.786105 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.786107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.786130 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:08:06.786204 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.786205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.786214 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.786262 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.786539 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.787002 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.787782 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.787987 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.788071 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.789867 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.789869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.789869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.789876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.791247 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.791705 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.791707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.791707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.791766 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.791969 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.791970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.791970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.792077 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.792151 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.792302 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.792383 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:06.792479 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.786035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.0.0.1:32768)
0923 22:08:06.786087 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.786098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.786146 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.786264 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.786264 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.786420 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.786460 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.786462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.787082 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.787083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.787682 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.787687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.788072 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.788073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.788142 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.788559 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.788564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.789775 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.791358 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.791359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.791360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.791532 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.791841 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.791841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.791842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.791905 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.792145 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.792146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.792193 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.792279 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.792384 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 22:08:06.793171 8089 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.244.252:38891: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07"
member_type: NON_VOTER
last_known_addr {
host: "127.1.244.252"
port: 38891
}
attrs {
promote: true
}
}
dest_uuid: "bd775ca101fd4cc2ad5778cbff713bcf"
cas_config_opid_index: 5
I20260923 22:08:06.793442 8089 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } }
I20260923 22:08:06.793771 8650 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 02c291ffca5f4e84a28d99b3947d4c07 (127.1.244.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.794530 8650 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.244.252:38891
I20260923 22:08:06.794608 8650 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.794679 8650 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:06.795291 8784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:06.796257 8784 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Refusing update from remote peer bd775ca101fd4cc2ad5778cbff713bcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 22:08:06.796537 8742 master.cc:537] Master@127.1.244.252:38891 shutting down...
I20260923 22:08:06.796586 8649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:08:06.797297 8074 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf -> Peer 02c291ffca5f4e84a28d99b3947d4c07 (127.1.244.252:38891): Couldn't send request to peer 02c291ffca5f4e84a28d99b3947d4c07. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:08:06.799912 8742 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:06.800006 8742 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:06.800062 8742 tablet_replica.cc:330] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: stopping tablet replica
I20260923 22:08:06.800643 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.794953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37550 (local address 127.1.244.253:41969)
0923 22:08:06.794993 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.794995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.795069 (+ 74us) server_negotiation.cc:407] Connection header received
0923 22:08:06.795162 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.795163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.795197 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.795241 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:06.795478 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.796203 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.799399 (+ 3196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.800043 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.800097 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.800173 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.800518 (+ 345us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.800560 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:08:06.800890 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.794904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.0.0.1:37550)
0923 22:08:06.795021 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.795032 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.795093 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.795322 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.795323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.795324 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:06.795361 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.795363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.796309 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.796310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.799292 (+ 2982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.799295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.800340 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.800341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.800406 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.800430 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.800758 (+ 328us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.800807 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260923 22:08:06.801144 8698 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 LEARNER]: Refusing update from remote peer bd775ca101fd4cc2ad5778cbff713bcf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 22:08:06.801618 8836 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260923 22:08:06.802186 8698 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 02c291ffca5f4e84a28d99b3947d4c07 (127.1.244.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.802654 8742 master.cc:552] Master@127.1.244.252:38891 shutdown complete.
I20260923 22:08:06.802697 8742 master_runner.cc:305] Clearing existing system tablet
I20260923 22:08:06.802826 8742 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.802872 8742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.803624 8742 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:06.803701 8727 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd775ca101fd4cc2ad5778cbff713bcf. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.803790 8727 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:08:06.803828 8727 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: true } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.803864 8727 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:08:06.804049 8836 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: attempting to promote NON_VOTER ae670858be364ed6bc4c88d32a1eced2 to VOTER
I20260923 22:08:06.804250 8836 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } }
I20260923 22:08:06.804819 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.804714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.0.0.1:32776)
0923 22:08:06.804754 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.804793 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 22:08:06.805051 8698 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 LEARNER]: Refusing update from remote peer bd775ca101fd4cc2ad5778cbff713bcf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260923 22:08:06.805294 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.805164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.0.0.1:32790)
0923 22:08:06.805247 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.805271 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260923 22:08:06.805378 8650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260923 22:08:06.805620 8074 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf -> Peer 02c291ffca5f4e84a28d99b3947d4c07 (127.1.244.252:38891): Couldn't send request to peer 02c291ffca5f4e84a28d99b3947d4c07. Status: Network error: Client connection negotiation failed: client connection to 127.1.244.252:38891: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 22:08:06.805842 8840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.805949 8742 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.805997 8742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260923 22:08:06.806020 8742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal
uuid: "02c291ffca5f4e84a28d99b3947d4c07"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "498c5ac7350005e1a15b5cd995cff08d"
server_key_iv: "c25ae0692276cab48ab722bdbce4548b"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.805982 8650 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ae670858be364ed6bc4c88d32a1eced2 (127.1.244.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.806811 8742 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:06.806799 8650 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.806880 8650 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:06.806934 8650 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.807005 8650 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:06.807268 8698 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ae670858be364ed6bc4c88d32a1eced2 (127.1.244.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.808166 8727 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.808254 8727 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:06.808405 8742 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 22:08:06.808465 8742 log.cc:1196] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260923 22:08:06.808657 8742 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Deleting consensus metadata
I20260923 22:08:06.808785 8742 master_runner.cc:315] Copying system tablet from 127.1.244.254:34767
I20260923 22:08:06.809442 8742 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: tablet copy: Beginning tablet copy session from remote peer at address 127.1.244.254:34767
I20260923 22:08:06.815879 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.809747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.1.244.254:34767)
0923 22:08:06.809787 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.809789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.809893 (+ 104us) server_negotiation.cc:407] Connection header received
0923 22:08:06.809951 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.809952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.809962 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.810017 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.810327 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.810828 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.811795 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.812003 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.812088 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.813082 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.813083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.813084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.813091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.814626 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.815039 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.815040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.815041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.815116 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.815344 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.815345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.815345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.815459 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.815530 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.815719 (+ 189us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.815786 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:08:06.816139 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.809718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:57946)
0923 22:08:06.809814 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.809823 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.809889 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.810024 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.810025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.810204 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.810249 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.810250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.810915 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.810916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.811717 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.811719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.812090 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.812091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.812154 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.812672 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.812677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.812992 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.814717 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.814718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.814718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.814959 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.815194 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.815195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.815196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.815266 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.815533 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.815534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.815610 (+ 76us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.815680 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.815786 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":49,"threads_started":1}
I20260923 22:08:06.816365 8110 tablet_copy_service.cc:140] P bd775ca101fd4cc2ad5778cbff713bcf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 02c291ffca5f4e84a28d99b3947d4c07 ({username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:57946)
I20260923 22:08:06.816417 8110 tablet_copy_service.cc:161] P bd775ca101fd4cc2ad5778cbff713bcf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 02c291ffca5f4e84a28d99b3947d4c07 at {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:57946: session id = 02c291ffca5f4e84a28d99b3947d4c07-00000000000000000000000000000000
I20260923 22:08:06.816850 8110 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 22:08:06.817301 8742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 22:08:06.818179 8742 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: tablet copy: Starting download of 0 data blocks...
I20260923 22:08:06.818333 8742 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: tablet copy: Starting download of 1 WAL segments...
I20260923 22:08:06.819034 8742 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 22:08:06.819773 8110 tablet_copy_service.cc:342] P bd775ca101fd4cc2ad5778cbff713bcf: Request end of tablet copy session 02c291ffca5f4e84a28d99b3947d4c07-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:57946
I20260923 22:08:06.819828 8110 tablet_copy_service.cc:433] P bd775ca101fd4cc2ad5778cbff713bcf: ending tablet copy session 02c291ffca5f4e84a28d99b3947d4c07-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 02c291ffca5f4e84a28d99b3947d4c07
I20260923 22:08:06.824366 8742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.825490 8847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.828374 8848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.828701 8742 server_base.cc:1034] running on GCE node
W20260923 22:08:06.831712 8850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.831872 8742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.832029 8742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.833148 8742 hybrid_clock.cc:648] HybridClock initialized: now 1790201286833127 us; error 43 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.834549 8742 init.cc:377] Logged in from keytab as oryx/127.1.244.252@KRBTEST.COM (short username oryx)
I20260923 22:08:06.834841 8742 webserver.cc:460] Webserver started at http://127.1.244.252:37837/ using document root <none> and password file <none>
I20260923 22:08:06.834962 8742 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.835000 8742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.835814 8742 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:06.836395 8857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.836570 8742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:06.836615 8742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal
uuid: "02c291ffca5f4e84a28d99b3947d4c07"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "498c5ac7350005e1a15b5cd995cff08d"
server_key_iv: "c25ae0692276cab48ab722bdbce4548b"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.836678 8742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.843989 8742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.845650 8742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.849073 8742 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:38891
I20260923 22:08:06.849455 8742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
I20260923 22:08:06.849733 8909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:38891 every 8 connection(s)
I20260923 22:08:06.850288 8910 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:08:06.850680 8910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Bootstrap starting.
I20260923 22:08:06.853104 8910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:08:06.853331 8910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Bootstrap complete.
I20260923 22:08:06.853528 8910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } }
I20260923 22:08:06.853615 8910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 02c291ffca5f4e84a28d99b3947d4c07, State: Initialized, Role: LEARNER
I20260923 22:08:06.853660 8910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } }
I20260923 22:08:06.853848 8910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:06.853816 8913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:06.854007 8913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:08:06.855268 8924 catalog_manager.cc:1259] Loaded cluster ID: 3f507a557c87407892d6df79cef3c379
I20260923 22:08:06.855362 8924 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: loading cluster ID for follower catalog manager: success
I20260923 22:08:06.856629 8924 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: acquiring CA information for follower catalog manager: success
I20260923 22:08:06.856988 8924 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260923 22:08:06.905645 8835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:06.905912 8835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:06.905951 8835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:06.907634 8835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:06.907692 8835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:06.907735 8835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:06.909864 8835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:33885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=35965
--tserver_master_addrs=127.1.244.254:34767,127.1.244.253:41969,127.1.244.252:38891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:06.910125 8835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:06.910403 8835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.916230 8932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.916277 8930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.916265 8929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.916713 8835 server_base.cc:1034] running on GCE node
I20260923 22:08:06.916918 8835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.917192 8835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.918360 8835 hybrid_clock.cc:648] HybridClock initialized: now 1790201286918341 us; error 33 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.921279 8835 init.cc:377] Logged in from keytab as oryx/127.1.244.193@KRBTEST.COM (short username oryx)
I20260923 22:08:06.921729 8835 webserver.cc:460] Webserver started at http://127.1.244.193:35965/ using document root <none> and password file <none>
I20260923 22:08:06.921959 8835 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.922043 8835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.923485 8835 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:06.924213 8939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.924409 8835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:08:06.924515 8835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "0ec38f76df844757bbcfec9a04b56e98"
format_stamp: "Formatted at 2026-09-23 22:08:05 on dist-test-slave-b5jz"
server_key: "94553e2e73acabe8eeafce05a9c6302b"
server_key_iv: "bff75873e6dddc936e23f1769a0d7c4d"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.924871 8835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.957119 8835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.959733 8835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:06.959901 8835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.960146 8835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:06.960479 8835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:06.960537 8835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.960583 8835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:06.960608 8835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.966946 8835 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:33885
I20260923 22:08:06.967365 8835 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:06.967469 9052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:33885 every 8 connection(s)
I20260923 22:08:06.968956 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 8835
I20260923 22:08:06.969125 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 8286
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260923 22:08:06.976679 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:39369
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:34767
--webserver_port=44903
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.244.254:34767,127.1.244.253:41969
--tserver_master_addrs=127.1.244.254:34767,127.1.244.253:41969,127.1.244.252:38891 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
Sep 23 22:08:06 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
I20260923 22:08:06.981668 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.968594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:59137 (local address 127.1.244.254:34767)
0923 22:08:06.968648 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.968650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.968686 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:08:06.969220 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.969223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.969237 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.969266 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.970729 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.971531 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.973155 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.973459 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.973499 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.978124 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.978126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.978126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.978135 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.980405 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.980880 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.980881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.980881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.980933 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.981128 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.981128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.981129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.981249 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.981290 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.981493 (+ 203us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.981576 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:08:06.981851 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.968642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.1.244.193:59691)
0923 22:08:06.968752 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.968760 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.969096 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.970703 (+ 1607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.970704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.970858 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.970926 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.970928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.971821 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.971822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.973049 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.973050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.973591 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.973593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.973761 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.974761 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.974764 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.977910 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.980085 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.980092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.980095 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.980409 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.980643 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.980643 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.980644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.980789 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.981130 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.981136 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.981303 (+ 167us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.981417 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.981596 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":44,"thread_start_us":70,"threads_started":1}
I20260923 22:08:06.981961 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.968149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:40181 (local address 127.1.244.252:38891)
0923 22:08:06.968283 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.968286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.968479 (+ 193us) server_negotiation.cc:407] Connection header received
0923 22:08:06.969634 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.969636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.969657 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.969703 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.970735 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.971884 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.973291 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.973664 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.973711 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.978421 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.978422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.978423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.978431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.980452 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.980995 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.980996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.980997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.981059 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.981276 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.981277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.981278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.981424 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.981474 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.981697 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.981796 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260923 22:08:06.981985 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.968477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.193:59137)
0923 22:08:06.968610 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.968621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.969105 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.969842 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.969843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.970298 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.970623 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.970631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.971677 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.971678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.973047 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.973050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.973592 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.973593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.973761 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.974697 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.974706 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.978009 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.980494 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.980495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.980495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.980793 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.981004 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.981004 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.981005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.981056 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.981349 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.981351 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.981384 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.981781 (+ 397us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.981877 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":73,"thread_start_us":63,"threads_started":1}
I20260923 22:08:06.982336 9054 heartbeater.cc:347] Connected to a master server at 127.1.244.254:34767
I20260923 22:08:06.982430 9054 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:06.982609 9054 heartbeater.cc:511] Master 127.1.244.254:34767 requested a full tablet report, sending...
I20260923 22:08:06.982818 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.968154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.1.244.193:40181)
0923 22:08:06.968316 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.968367 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.969092 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.969787 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.969792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.970293 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.970623 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.970629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.971981 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.971982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.973050 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.973052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.973793 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.973793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.973879 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.974829 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.974835 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.978320 (+ 3485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.980574 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.980576 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.980576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.980898 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.981139 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.981140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.981140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.981203 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.981529 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.981530 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.981580 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.982539 (+ 959us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.982660 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":8,"thread_start_us":85,"threads_started":1}
I20260923 22:08:06.983040 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.968835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:59691 (local address 127.1.244.253:41969)
0923 22:08:06.968893 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.968895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.968903 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:06.970591 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.970593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.970605 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.970638 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.971020 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.971739 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.973155 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.973456 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.973499 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.978024 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.978026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.978026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.978034 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.979985 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.980495 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.980496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.980497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.980573 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.980876 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.980877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.980878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.981044 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.981107 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.982850 (+ 1743us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.982942 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:08:06.983211 9053 heartbeater.cc:347] Connected to a master server at 127.1.244.252:38891
I20260923 22:08:06.983276 9053 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:06.983359 9053 heartbeater.cc:511] Master 127.1.244.252:38891 requested a full tablet report, sending...
I20260923 22:08:06.983453 8089 ts_manager.cc:194] Re-registered known tserver with Master: 0ec38f76df844757bbcfec9a04b56e98 (127.1.244.193:33885)
I20260923 22:08:06.983786 8874 ts_manager.cc:194] Registered new tserver with Master: 0ec38f76df844757bbcfec9a04b56e98 (127.1.244.193:33885)
I20260923 22:08:06.983928 8089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:59137
I20260923 22:08:06.984222 9055 heartbeater.cc:347] Connected to a master server at 127.1.244.253:41969
I20260923 22:08:06.984272 9055 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:06.984349 9055 heartbeater.cc:511] Master 127.1.244.253:41969 requested a full tablet report, sending...
I20260923 22:08:06.984670 8688 ts_manager.cc:194] Registered new tserver with Master: 0ec38f76df844757bbcfec9a04b56e98 (127.1.244.193:33885)
W20260923 22:08:07.062852 9061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:07.063050 9061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:07.063072 9061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:07.064296 9061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:07.064337 9061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:07.064363 9061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:07.065852 9061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:39369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=44903
--tserver_master_addrs=127.1.244.254:34767,127.1.244.253:41969,127.1.244.252:38891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:07.066118 9061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:07.066401 9061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:07.071504 9069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:07.071703 9067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:07.071835 9066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:07.071916 9061 server_base.cc:1034] running on GCE node
I20260923 22:08:07.072122 9061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:07.072400 9061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:07.073570 9061 hybrid_clock.cc:648] HybridClock initialized: now 1790201287073560 us; error 42 us; skew 500 ppm
Sep 23 22:08:07 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:07.076313 9061 init.cc:377] Logged in from keytab as oryx/127.1.244.194@KRBTEST.COM (short username oryx)
I20260923 22:08:07.076808 9061 webserver.cc:460] Webserver started at http://127.1.244.194:44903/ using document root <none> and password file <none>
I20260923 22:08:07.077103 9061 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:07.077191 9061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:07.078497 9061 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:07.079084 9076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:07.079253 9061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:07.079339 9061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "5567facef3cf4b4f845f9af30f43580b"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "a211a4b0f52aaacb27822a9da8c73be9"
server_key_iv: "7981dd5d2aaff7e2d76a75b4361ed6d6"
server_key_version: "encryptionkey@0"
I20260923 22:08:07.079718 9061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:07.106860 9061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:07.108798 9061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:07.108909 9061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:07.109084 9061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:07.109324 9061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:07.109352 9061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:07.109383 9061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:07.109396 9061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:07.115548 9061 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:39369
I20260923 22:08:07.115614 9189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:39369 every 8 connection(s)
I20260923 22:08:07.115994 9061 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:07.122629 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 9061
I20260923 22:08:07.122782 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 8420
Sep 23 22:08:07 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
Sep 23 22:08:07 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
Sep 23 22:08:07 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260923 22:08:07.128307 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.116718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:51175 (local address 127.1.244.253:41969)
0923 22:08:07.116764 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.116766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.117167 (+ 401us) server_negotiation.cc:407] Connection header received
0923 22:08:07.118151 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.118152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.118164 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.118230 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.119080 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.119863 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.121045 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.121339 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.121457 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.124952 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.124953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.124954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.124961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.126731 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.127268 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.127269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.127270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.127322 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.127594 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.127595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.127595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.127732 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.127772 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.128130 (+ 358us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.128210 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:08:07.128306 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.117044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:37801 (local address 127.1.244.254:34767)
0923 22:08:07.117090 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.117093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.118280 (+ 1187us) server_negotiation.cc:407] Connection header received
0923 22:08:07.118381 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.118382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.118392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.118434 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.119080 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.119863 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.121021 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.121239 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.121271 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.124025 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.124026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.124026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.124033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.126642 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.127221 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.127222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.127222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.127281 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.127570 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.127571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.127571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.127704 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.127752 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.128130 (+ 378us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.128210 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:08:07.128357 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.116658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:60195 (local address 127.1.244.252:38891)
0923 22:08:07.116727 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.116729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.117003 (+ 274us) server_negotiation.cc:407] Connection header received
0923 22:08:07.118151 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.118152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.118164 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.118230 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.119074 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.119633 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.120636 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.120840 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.120932 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.124874 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.124876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.124876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.124883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.126596 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.127203 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.127204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.127205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.127266 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.127560 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.127561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.127561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.127716 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.127762 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.128115 (+ 353us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.128212 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:08:07.128866 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.116692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.1.244.194:51175)
0923 22:08:07.117093 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.117102 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.118043 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.118676 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.118677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.118852 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:07.118970 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.118977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.119962 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.119963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.120919 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.120921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.121459 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.121460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.121530 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.122036 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:07.122041 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:07.124837 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:07.126807 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.126808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.126808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.127190 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.127435 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.127436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.127436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.127485 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.127836 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:07.127837 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:07.127969 (+ 132us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.128392 (+ 423us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.128682 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":130,"spinlock_wait_cycles":15232,"thread_start_us":115,"threads_started":1}
I20260923 22:08:07.128871 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.116679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.1.244.194:60195)
0923 22:08:07.116865 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.116906 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.118036 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.118304 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.118308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.118755 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:07.118970 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.118975 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.119731 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.119732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.120528 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.120530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.120954 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.120955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.121126 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.121712 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:07.121719 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:07.124774 (+ 3055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:07.126686 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.126693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.126695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.127116 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.127335 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.127336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.127337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.127464 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.127808 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:07.127814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:07.127968 (+ 154us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.128434 (+ 466us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.128610 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":3,"thread_start_us":113,"threads_started":1}
I20260923 22:08:07.129415 9192 heartbeater.cc:347] Connected to a master server at 127.1.244.253:41969
I20260923 22:08:07.129515 9192 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:07.129706 9192 heartbeater.cc:511] Master 127.1.244.253:41969 requested a full tablet report, sending...
I20260923 22:08:07.129801 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.116979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.194:37801)
0923 22:08:07.118224 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.118233 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.118305 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.118439 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.118440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.118745 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:07.118971 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.118977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.119962 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.119963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.120920 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.120921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.121370 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.121371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.121432 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.121862 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:07.121867 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:07.123931 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:07.126742 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.126743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.126744 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.127130 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.127357 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.127357 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.127357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.127469 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.127804 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:07.127810 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:07.127969 (+ 159us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.128903 (+ 934us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.129536 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1220,"mutex_wait_us":85,"thread_start_us":72,"threads_started":1}
I20260923 22:08:07.130180 8688 ts_manager.cc:194] Registered new tserver with Master: 5567facef3cf4b4f845f9af30f43580b (127.1.244.194:39369)
I20260923 22:08:07.130204 9190 heartbeater.cc:347] Connected to a master server at 127.1.244.252:38891
I20260923 22:08:07.130257 9190 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:07.130303 9191 heartbeater.cc:347] Connected to a master server at 127.1.244.254:34767
I20260923 22:08:07.130324 9190 heartbeater.cc:511] Master 127.1.244.252:38891 requested a full tablet report, sending...
I20260923 22:08:07.130355 9191 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:07.130440 9191 heartbeater.cc:511] Master 127.1.244.254:34767 requested a full tablet report, sending...
I20260923 22:08:07.130584 8874 ts_manager.cc:194] Registered new tserver with Master: 5567facef3cf4b4f845f9af30f43580b (127.1.244.194:39369)
I20260923 22:08:07.130666 8089 ts_manager.cc:194] Re-registered known tserver with Master: 5567facef3cf4b4f845f9af30f43580b (127.1.244.194:39369)
I20260923 22:08:07.131191 8089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:37801
I20260923 22:08:07.131420 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:42541
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:34767
--webserver_port=41857
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.244.254:34767,127.1.244.253:41969
--tserver_master_addrs=127.1.244.254:34767,127.1.244.253:41969,127.1.244.252:38891 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:07.212388 9197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:07.212591 9197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:07.212613 9197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:07.213903 9197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:07.213943 9197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:07.213969 9197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:07.215664 9197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:42541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=41857
--tserver_master_addrs=127.1.244.254:34767,127.1.244.253:41969,127.1.244.252:38891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:07.215917 9197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:07.216194 9197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:07.221231 9197 server_base.cc:1034] running on GCE node
W20260923 22:08:07.221215 9203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:07.221215 9205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:07.221225 9202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:07.221741 9197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:07.221994 9197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:07.223160 9197 hybrid_clock.cc:648] HybridClock initialized: now 1790201287223150 us; error 39 us; skew 500 ppm
Sep 23 22:08:07 dist-test-slave-b5jz krb5kdc[8045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:07.225811 9197 init.cc:377] Logged in from keytab as oryx/127.1.244.195@KRBTEST.COM (short username oryx)
I20260923 22:08:07.226269 9197 webserver.cc:460] Webserver started at http://127.1.244.195:41857/ using document root <none> and password file <none>
I20260923 22:08:07.226482 9197 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:07.226537 9197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:07.227907 9197 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:07.228483 9212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:07.228719 9197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:07.228802 9197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "e58c1112264e4386a4cce33487fe2dc3"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-b5jz"
server_key: "93f2412032ce6038a1173d61caac52e5"
server_key_iv: "878527868e857696d82bd39eed319a85"
server_key_version: "encryptionkey@0"
I20260923 22:08:07.229131 9197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:07.247771 9197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:07.249578 9197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:07.249688 9197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:07.249876 9197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:07.250186 9197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:07.250221 9197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:07.250267 9197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:07.250295 9197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:07.256862 9197 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:42541
I20260923 22:08:07.256955 9325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:42541 every 8 connection(s)
I20260923 22:08:07.257325 9197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:07 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
Sep 23 22:08:07 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
Sep 23 22:08:07 dist-test-slave-b5jz krb5kdc[8045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:07.266413 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 9197
I20260923 22:08:07.268694 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.258035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:54785 (local address 127.1.244.252:38891)
0923 22:08:07.258084 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.258087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.258405 (+ 318us) server_negotiation.cc:407] Connection header received
0923 22:08:07.259466 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.259467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.259480 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.259533 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.260357 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.260860 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.262233 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.262445 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.262522 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.265062 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.265063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.265063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.265071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.266778 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.267309 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.267311 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.267311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.267371 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.267696 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.267697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.267697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.267848 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.267912 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.268509 (+ 597us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.268597 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:08:07.269142 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.258609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:56277 (local address 127.1.244.254:34767)
0923 22:08:07.258663 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.258664 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.258680 (+ 16us) server_negotiation.cc:407] Connection header received
0923 22:08:07.259611 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.259613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.259624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.259679 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.260370 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.261135 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.262063 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.262375 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.262439 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.266095 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.266096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.266096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.266106 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.267894 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.268265 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.268266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.268266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.268322 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.268545 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.268546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.268546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.268672 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.268715 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.268983 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.269054 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:08:07.269244 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.258068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.1.244.195:54785)
0923 22:08:07.258262 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.258306 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.259340 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.259630 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.259634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.260059 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:07.260259 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.260264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.260971 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.260972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.262118 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.262120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.262520 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.262521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.262641 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.263309 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:07.263320 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:07.264963 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:07.266887 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.266893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.266897 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.267213 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.267451 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.267452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.267452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.267610 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.268151 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:07.268158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:07.268330 (+ 172us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.268814 (+ 484us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.269013 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":119,"threads_started":1}
I20260923 22:08:07.269475 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.258395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:56277)
0923 22:08:07.258513 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.258522 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.259361 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.259707 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.259707 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.260055 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:07.260260 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.260266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.261220 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.261221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.261960 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.261962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.262506 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.262507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.262643 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.263437 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:07.263441 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:07.265994 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:07.267974 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.267975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.267975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.268183 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.268408 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.268408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.268409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.268472 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.268816 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:07.268816 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:07.268863 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.269254 (+ 391us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.269364 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":7,"thread_start_us":65,"threads_started":1}
I20260923 22:08:07.269476 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.258517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:33227 (local address 127.1.244.253:41969)
0923 22:08:07.258589 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.258591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.258599 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:07.259732 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.259733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.259743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.259792 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.260379 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.261117 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.262203 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.262499 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.262558 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.266379 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.266380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.266381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.266390 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.268198 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.268611 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.268612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.268613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.268673 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.268908 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.268909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.268909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.269033 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.269080 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.269300 (+ 220us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.269392 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:08:07.269666 9326 heartbeater.cc:347] Connected to a master server at 127.1.244.252:38891
I20260923 22:08:07.269771 9326 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:07.269966 9326 heartbeater.cc:511] Master 127.1.244.252:38891 requested a full tablet report, sending...
I20260923 22:08:07.270103 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.258068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.1.244.195:33227)
0923 22:08:07.258262 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.258306 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.259359 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.259810 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.259811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.260075 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:07.260260 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.260266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.261201 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.261201 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.262123 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.262124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.262606 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.262607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.262657 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.263363 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:07.263368 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:07.266273 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:07.268296 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.268297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.268298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.268523 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.268753 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:07.268753 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:07.268754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:07.268819 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:07.269129 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:07.269130 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:07.269173 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.269760 (+ 587us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.269864 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":75,"thread_start_us":110,"threads_started":1}
I20260923 22:08:07.270135 9327 heartbeater.cc:347] Connected to a master server at 127.1.244.254:34767
I20260923 22:08:07.270172 9327 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:07.270251 9327 heartbeater.cc:511] Master 127.1.244.254:34767 requested a full tablet report, sending...
I20260923 22:08:07.270476 9328 heartbeater.cc:347] Connected to a master server at 127.1.244.253:41969
I20260923 22:08:07.270533 9328 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:07.270603 9328 heartbeater.cc:511] Master 127.1.244.253:41969 requested a full tablet report, sending...
I20260923 22:08:07.270612 8874 ts_manager.cc:194] Registered new tserver with Master: e58c1112264e4386a4cce33487fe2dc3 (127.1.244.195:42541)
I20260923 22:08:07.270810 8089 ts_manager.cc:194] Re-registered known tserver with Master: e58c1112264e4386a4cce33487fe2dc3 (127.1.244.195:42541)
I20260923 22:08:07.270936 8688 ts_manager.cc:194] Registered new tserver with Master: e58c1112264e4386a4cce33487fe2dc3 (127.1.244.195:42541)
I20260923 22:08:07.271320 8089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:56277
I20260923 22:08:07.278591 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.267578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.1.244.254:34767)
0923 22:08:07.269893 (+ 2315us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.269895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.269902 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:07.269921 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.269922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.269933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.269999 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.271086 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.271861 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.272943 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.273256 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.273326 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.274898 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.274900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.274900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.274908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.277069 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.277517 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.277518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.277518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.277580 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.277795 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.277796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.277796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.277998 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.278103 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.278394 (+ 291us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.278480 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1585,"thread_start_us":89,"threads_started":1}
I20260923 22:08:07.278605 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.268088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37552 (local address 127.1.244.253:41969)
0923 22:08:07.268713 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.268715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.271237 (+ 2522us) server_negotiation.cc:407] Connection header received
0923 22:08:07.271258 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.271258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.271269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.271325 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.272080 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.272636 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.273646 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.273951 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.274015 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.275344 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.275346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.275346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.275353 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.277191 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.277628 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.277630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.277630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.277682 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.277895 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.277896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.277897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.278030 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.278127 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.278299 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.278486 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":49,"threads_started":1}
I20260923 22:08:07.278896 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.268477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.1.244.252:38891)
0923 22:08:07.268787 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.268789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.271455 (+ 2666us) server_negotiation.cc:407] Connection header received
0923 22:08:07.271565 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.271566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.271576 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.271606 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.272302 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.272989 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.273920 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.274170 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.274471 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.275797 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.275798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.275799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.275805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.277520 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.277867 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.277869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.277869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.277922 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.278151 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.278152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.278152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.278300 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.278397 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.278597 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.278790 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":54,"threads_started":1}
I20260923 22:08:07.280417 8089 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:07.281983 8089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:07.289230 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.285632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.1.244.195:42541)
0923 22:08:07.285911 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.285917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.285930 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:08:07.285972 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.285974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.285993 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.286232 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:07.286292 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.287028 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.288163 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.288899 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.288994 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.289063 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.289118 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.289157 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":86,"threads_started":1}
I20260923 22:08:07.289216 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.285516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42541 (local address 127.0.0.1:47232)
0923 22:08:07.285565 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.285578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.285626 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.286106 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.286107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.286109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:07.286158 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.286160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.287114 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.287115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.288072 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.288074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.288995 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.288995 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.289049 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.289052 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.289102 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.289143 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:08:07.289687 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.285779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:33885 (local address 127.0.0.1:47072)
0923 22:08:07.285876 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.285884 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.285921 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.286346 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.286347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.286349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:07.286385 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.286386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.287620 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.287621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.288552 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.288554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.289488 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.289489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.289553 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.289558 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.289608 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.289638 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":71,"thread_start_us":56,"threads_started":1}
I20260923 22:08:07.290004 9260 tablet_service.cc:1467] Processing CreateTablet for tablet 8cad3de6ad5f4f9d80c8d701623de87a (DEFAULT_TABLE table=test-table [id=9cd6ef489ca8410c8465784a598f7a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:07.290236 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.285609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:39369 (local address 127.0.0.1:55290)
0923 22:08:07.285731 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.285740 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.285778 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.286249 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.286250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.286252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:07.286297 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.286298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.287603 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.287604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.288881 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.288885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.289965 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.289966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.290021 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.290043 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.290109 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.290159 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":89,"thread_start_us":85,"threads_started":1}
I20260923 22:08:07.290248 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.285702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55290 (local address 127.1.244.194:39369)
0923 22:08:07.286069 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.286073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.286083 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:07.286125 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.286127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.286142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.286240 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:07.286361 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.287467 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.288976 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.289875 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.289952 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.290039 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.290126 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.290166 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":78,"threads_started":1}
I20260923 22:08:07.290431 9260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cad3de6ad5f4f9d80c8d701623de87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:07.290832 9124 tablet_service.cc:1467] Processing CreateTablet for tablet 8cad3de6ad5f4f9d80c8d701623de87a (DEFAULT_TABLE table=test-table [id=9cd6ef489ca8410c8465784a598f7a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:07.291082 9124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cad3de6ad5f4f9d80c8d701623de87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:07.292348 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.285971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47072 (local address 127.1.244.193:33885)
0923 22:08:07.286149 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.286152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.286161 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:07.286198 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.286200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.286217 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.286287 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:07.286468 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.287526 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.288637 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.289258 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.289479 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.292176 (+ 2697us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.292210 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.292270 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
I20260923 22:08:07.293131 8987 tablet_service.cc:1467] Processing CreateTablet for tablet 8cad3de6ad5f4f9d80c8d701623de87a (DEFAULT_TABLE table=test-table [id=9cd6ef489ca8410c8465784a598f7a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:07.293453 8987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cad3de6ad5f4f9d80c8d701623de87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:07.294246 9352 tablet_bootstrap.cc:492] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: Bootstrap starting.
I20260923 22:08:07.294600 9353 tablet_bootstrap.cc:492] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: Bootstrap starting.
I20260923 22:08:07.294967 9352 tablet_bootstrap.cc:654] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:07.295203 9353 tablet_bootstrap.cc:654] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:07.295288 9352 log.cc:824] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:07.295511 9353 log.cc:824] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:07.296106 9352 tablet_bootstrap.cc:492] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: No bootstrap required, opened a new log
I20260923 22:08:07.296190 9352 ts_tablet_manager.cc:1397] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:07.296312 9353 tablet_bootstrap.cc:492] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: No bootstrap required, opened a new log
I20260923 22:08:07.296384 9353 ts_tablet_manager.cc:1397] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:07.296980 9356 tablet_bootstrap.cc:492] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: Bootstrap starting.
I20260923 22:08:07.297650 9356 tablet_bootstrap.cc:654] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:07.298457 9356 log.cc:824] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:07.298519 9353 raft_consensus.cc:348] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } }
I20260923 22:08:07.298751 9353 raft_consensus.cc:374] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:07.298804 9353 raft_consensus.cc:729] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5567facef3cf4b4f845f9af30f43580b, State: Initialized, Role: FOLLOWER
I20260923 22:08:07.298950 9353 consensus_queue.cc:260] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } }
I20260923 22:08:07.299243 9353 ts_tablet_manager.cc:1428] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:08:07.299273 9352 raft_consensus.cc:348] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } }
I20260923 22:08:07.299396 9191 heartbeater.cc:503] Master 127.1.244.254:34767 was elected leader, sending a full tablet report...
I20260923 22:08:07.299436 9352 raft_consensus.cc:374] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:07.299471 9352 raft_consensus.cc:729] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58c1112264e4386a4cce33487fe2dc3, State: Initialized, Role: FOLLOWER
I20260923 22:08:07.299626 9352 consensus_queue.cc:260] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } }
I20260923 22:08:07.299625 9356 tablet_bootstrap.cc:492] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: No bootstrap required, opened a new log
I20260923 22:08:07.300172 9356 ts_tablet_manager.cc:1397] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:08:07.300352 9352 ts_tablet_manager.cc:1428] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260923 22:08:07.300386 9327 heartbeater.cc:503] Master 127.1.244.254:34767 was elected leader, sending a full tablet report...
I20260923 22:08:07.302121 9356 raft_consensus.cc:348] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } }
I20260923 22:08:07.302241 9356 raft_consensus.cc:374] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:07.302263 9356 raft_consensus.cc:729] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec38f76df844757bbcfec9a04b56e98, State: Initialized, Role: FOLLOWER
I20260923 22:08:07.302336 9356 consensus_queue.cc:260] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } }
I20260923 22:08:07.302531 9356 ts_tablet_manager.cc:1428] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:07.302610 9054 heartbeater.cc:503] Master 127.1.244.254:34767 was elected leader, sending a full tablet report...
I20260923 22:08:07.320142 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.317581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.1.244.252:38891)
0923 22:08:07.317649 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.317651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.317661 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:07.317719 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.317720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.317749 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.317821 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:07.317962 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.318446 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.319355 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.319812 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.319903 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.319986 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.320031 (+ 45us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.320065 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:08:07.320142 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.317508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.0.0.1:32800)
0923 22:08:07.317577 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.317588 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.317654 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.317825 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.317826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.317827 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:07.317890 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.317892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.318535 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.318537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.319253 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.319255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.319902 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.319903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.319961 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.319965 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.320023 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.320072 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 22:08:07.320533 8877 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260923 22:08:07.321419 8913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd775ca101fd4cc2ad5778cbff713bcf. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:07.321532 8913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: This master's current role is: LEARNER
W20260923 22:08:07.366690 9193 tablet.cc:2369] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:08:07.468282 9056 tablet.cc:2369] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:08:07.508093 9329 tablet.cc:2369] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:07.562258 9360 raft_consensus.cc:482] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:07.562441 9360 raft_consensus.cc:504] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } }
I20260923 22:08:07.562837 9360 leader_election.cc:288] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e58c1112264e4386a4cce33487fe2dc3 (127.1.244.195:42541), 5567facef3cf4b4f845f9af30f43580b (127.1.244.194:39369)
I20260923 22:08:07.566768 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.563161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:38319 (local address 127.1.244.195:42541)
0923 22:08:07.563229 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.563244 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.563258 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:08:07.563330 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.563331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.563341 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.563413 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:07.563633 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.564407 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.565923 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.566429 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.566486 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.566493 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.566660 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.566699 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:08:07.566776 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.563059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42541 (local address 127.1.244.193:38319)
0923 22:08:07.563179 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.563191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.563249 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.563439 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.563440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.563443 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:07.563506 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.563508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.564509 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.564510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.565828 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.565831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.566511 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.566512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.566593 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.566597 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.566654 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.566699 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":45,"thread_start_us":69,"threads_started":1}
I20260923 22:08:07.566843 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.563332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:44571 (local address 127.1.244.194:39369)
0923 22:08:07.563388 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.563390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.563402 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:08:07.563512 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.563513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.563520 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.563650 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:07.563769 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.564447 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.565949 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.566511 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.566589 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.566598 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.566733 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.566775 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:08:07.567145 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.563248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:39369 (local address 127.1.244.193:44571)
0923 22:08:07.563337 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:07.563357 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:07.563432 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:07.563658 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:07.563658 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:07.563665 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:07.563706 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.563707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.564516 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.564516 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.565828 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:07.565831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.566588 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.566589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:07.566648 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.566651 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:07.566714 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:08:07.566760 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":50,"thread_start_us":37,"threads_started":1}
I20260923 22:08:07.567239 9280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cad3de6ad5f4f9d80c8d701623de87a" candidate_uuid: "0ec38f76df844757bbcfec9a04b56e98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58c1112264e4386a4cce33487fe2dc3" is_pre_election: true
I20260923 22:08:07.567406 9280 raft_consensus.cc:2454] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec38f76df844757bbcfec9a04b56e98 in term 0.
I20260923 22:08:07.567479 9144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cad3de6ad5f4f9d80c8d701623de87a" candidate_uuid: "0ec38f76df844757bbcfec9a04b56e98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5567facef3cf4b4f845f9af30f43580b" is_pre_election: true
I20260923 22:08:07.567651 9144 raft_consensus.cc:2454] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec38f76df844757bbcfec9a04b56e98 in term 0.
I20260923 22:08:07.567631 8943 leader_election.cc:302] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec38f76df844757bbcfec9a04b56e98, e58c1112264e4386a4cce33487fe2dc3; no voters:
I20260923 22:08:07.567761 9360 raft_consensus.cc:2781] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:07.567812 9360 raft_consensus.cc:482] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:07.567839 9360 raft_consensus.cc:3037] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:07.568491 9360 raft_consensus.cc:504] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } }
I20260923 22:08:07.568620 9360 leader_election.cc:288] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [CANDIDATE]: Term 1 election: Requested vote from peers e58c1112264e4386a4cce33487fe2dc3 (127.1.244.195:42541), 5567facef3cf4b4f845f9af30f43580b (127.1.244.194:39369)
I20260923 22:08:07.568807 9144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cad3de6ad5f4f9d80c8d701623de87a" candidate_uuid: "0ec38f76df844757bbcfec9a04b56e98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5567facef3cf4b4f845f9af30f43580b"
I20260923 22:08:07.568904 9280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cad3de6ad5f4f9d80c8d701623de87a" candidate_uuid: "0ec38f76df844757bbcfec9a04b56e98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58c1112264e4386a4cce33487fe2dc3"
I20260923 22:08:07.568917 9144 raft_consensus.cc:3037] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:07.568984 9280 raft_consensus.cc:3037] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:07.569828 9144 raft_consensus.cc:2454] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec38f76df844757bbcfec9a04b56e98 in term 1.
I20260923 22:08:07.569867 9280 raft_consensus.cc:2454] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec38f76df844757bbcfec9a04b56e98 in term 1.
I20260923 22:08:07.570001 8943 leader_election.cc:302] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec38f76df844757bbcfec9a04b56e98, 5567facef3cf4b4f845f9af30f43580b; no voters:
I20260923 22:08:07.570091 9360 raft_consensus.cc:2781] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:07.570199 9360 raft_consensus.cc:686] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 1 LEADER]: Becoming Leader. State: Replica: 0ec38f76df844757bbcfec9a04b56e98, State: Running, Role: LEADER
I20260923 22:08:07.570299 9360 consensus_queue.cc:237] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } }
I20260923 22:08:07.571113 8089 catalog_manager.cc:5495] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec38f76df844757bbcfec9a04b56e98 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "0ec38f76df844757bbcfec9a04b56e98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } health_report { overall_health: HEALTHY } } }
I20260923 22:08:07.571732 9369 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: true } } }
I20260923 22:08:07.571854 9369 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:07.572364 9369 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: attempting to promote NON_VOTER 02c291ffca5f4e84a28d99b3947d4c07 to VOTER
I20260923 22:08:07.572532 9369 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } }
I20260923 22:08:07.573012 8698 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Refusing update from remote peer bd775ca101fd4cc2ad5778cbff713bcf: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260923 22:08:07.573061 8877 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 LEARNER]: Refusing update from remote peer bd775ca101fd4cc2ad5778cbff713bcf: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260923 22:08:07.573227 9371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260923 22:08:07.573328 8649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260923 22:08:07.573894 9371 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 02c291ffca5f4e84a28d99b3947d4c07 (127.1.244.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:07.574107 8698 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 02c291ffca5f4e84a28d99b3947d4c07 (127.1.244.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:07.574126 8877 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 02c291ffca5f4e84a28d99b3947d4c07 (127.1.244.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:07.574750 9371 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:07.574824 9371 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:07.574908 9371 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:07.574955 9371 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:07.575038 8649 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:07.575095 8649 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:07.575057 9372 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:07.575129 8913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:07.575207 9372 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:07.575207 8913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:07.640601 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.638079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.1.244.193:33885)
0923 22:08:07.638131 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.638133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.638143 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:07.638207 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.638208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.638231 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.638317 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:07.638479 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.638961 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.639748 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.639967 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.640053 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.640238 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:07.640363 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:07.640454 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.640495 (+ 41us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.640516 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:08:07.642417 9280 raft_consensus.cc:1261] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [term 1 FOLLOWER]: Refusing update from remote peer 0ec38f76df844757bbcfec9a04b56e98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:07.642472 9144 raft_consensus.cc:1261] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [term 1 FOLLOWER]: Refusing update from remote peer 0ec38f76df844757bbcfec9a04b56e98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:07.642782 9367 consensus_queue.cc:1035] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:07.642938 9360 consensus_queue.cc:1035] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:07.645313 9374 mvcc.cc:205] Tried to move back new op lower bound from 7332664474180603904 to 7332664473888546816. Current Snapshot: MvccSnapshot[applied={T|T < 7332664474180603904}]
I20260923 22:08:07.646266 9375 mvcc.cc:205] Tried to move back new op lower bound from 7332664474180603904 to 7332664473888546816. Current Snapshot: MvccSnapshot[applied={T|T < 7332664474180603904}]
I20260923 22:08:07.646386 9376 mvcc.cc:205] Tried to move back new op lower bound from 7332664474180603904 to 7332664473888546816. Current Snapshot: MvccSnapshot[applied={T|T < 7332664474180603904}]
I20260923 22:08:07.648138 8090 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:07.648236 8090 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:07.649585 8090 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd775ca101fd4cc2ad5778cbff713bcf: Sending DeleteTablet for 3 replicas of tablet 8cad3de6ad5f4f9d80c8d701623de87a
I20260923 22:08:07.649899 9260 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cad3de6ad5f4f9d80c8d701623de87a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:07 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:47232
I20260923 22:08:07.649914 9124 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cad3de6ad5f4f9d80c8d701623de87a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:07 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:55290
I20260923 22:08:07.649943 8987 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cad3de6ad5f4f9d80c8d701623de87a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:07 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:47072
I20260923 22:08:07.650135 9384 tablet_replica.cc:330] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: stopping tablet replica
I20260923 22:08:07.650233 9384 raft_consensus.cc:2229] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:07.650328 9384 raft_consensus.cc:2258] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:07.650480 9385 tablet_replica.cc:330] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: stopping tablet replica
I20260923 22:08:07.650575 9385 raft_consensus.cc:2229] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:07.650677 9385 raft_consensus.cc:2258] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:07.651156 9385 ts_tablet_manager.cc:1905] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:07.652339 9383 tablet_replica.cc:330] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: stopping tablet replica
I20260923 22:08:07.652514 9383 raft_consensus.cc:2229] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:07.652580 9384 ts_tablet_manager.cc:1905] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:07.652706 9383 raft_consensus.cc:2258] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:07.653198 9383 ts_tablet_manager.cc:1905] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:07.653362 9385 ts_tablet_manager.cc:1918] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:07.653441 9385 log.cc:1196] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/8cad3de6ad5f4f9d80c8d701623de87a
I20260923 22:08:07.653715 9385 ts_tablet_manager.cc:1939] T 8cad3de6ad5f4f9d80c8d701623de87a P e58c1112264e4386a4cce33487fe2dc3: Deleting consensus metadata
I20260923 22:08:07.654273 9384 ts_tablet_manager.cc:1918] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:07.654351 9384 log.cc:1196] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/8cad3de6ad5f4f9d80c8d701623de87a
I20260923 22:08:07.654646 9384 ts_tablet_manager.cc:1939] T 8cad3de6ad5f4f9d80c8d701623de87a P 0ec38f76df844757bbcfec9a04b56e98: Deleting consensus metadata
I20260923 22:08:07.655210 9383 ts_tablet_manager.cc:1918] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:07.655321 9383 log.cc:1196] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/8cad3de6ad5f4f9d80c8d701623de87a
I20260923 22:08:07.655673 9383 ts_tablet_manager.cc:1939] T 8cad3de6ad5f4f9d80c8d701623de87a P 5567facef3cf4b4f845f9af30f43580b: Deleting consensus metadata
I20260923 22:08:07.655812 8077 catalog_manager.cc:4841] TS 0ec38f76df844757bbcfec9a04b56e98 (127.1.244.193:33885): tablet 8cad3de6ad5f4f9d80c8d701623de87a (table test-table [id=9cd6ef489ca8410c8465784a598f7a08]) successfully deleted
I20260923 22:08:07.655908 8077 catalog_manager.cc:4841] TS e58c1112264e4386a4cce33487fe2dc3 (127.1.244.195:42541): tablet 8cad3de6ad5f4f9d80c8d701623de87a (table test-table [id=9cd6ef489ca8410c8465784a598f7a08]) successfully deleted
I20260923 22:08:07.656077 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 8058
W20260923 22:08:07.662335 9079 connection.cc:535] server connection from 127.0.0.1:55290 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:08:07.669727 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.651152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.1.244.252:38891)
0923 22:08:07.651192 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.651194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.651202 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:07.651224 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.651225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.651234 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.651289 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.654996 (+ 3707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.655639 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.657995 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.658183 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.658273 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.665857 (+ 7584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.665858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.665859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.665867 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.667654 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.668066 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.668068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.668068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.668127 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.668905 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.668906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.668906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.669059 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.669162 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.669542 (+ 380us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.669616 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:07.672349 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.664432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.1.244.253:41969)
0923 22:08:07.664475 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.664478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.664485 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:07.664507 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.664508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.664520 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.664595 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.665657 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.666457 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.667628 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.667957 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.668007 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.669429 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.669430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.669430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.669437 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.671090 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.671492 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.671493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.671494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.671569 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.671779 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.671780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.671780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.671912 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.671980 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.672154 (+ 174us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.672227 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:07.673036 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:07.664164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.1.244.252:38891)
0923 22:08:07.664230 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:07.664232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:07.664342 (+ 110us) server_negotiation.cc:407] Connection header received
0923 22:08:07.664443 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:07.664444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:07.664456 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:07.664494 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:07.666081 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.666872 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.667967 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:07.668288 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:07.668327 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:07.669712 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:07.669713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:07.669713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:07.669720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:07.671814 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.672207 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.672208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.672208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.672265 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:07.672461 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:07.672462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:07.672462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:07.672580 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:07.672648 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:07.672869 (+ 221us) server_negotiation.cc:299] Negotiation successful
0923 22:08:07.672935 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:08:08.647524 9394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:08.647220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.194:48907)
0923 22:08:08.647368 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:08.647462 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260923 22:08:08.647733 9395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:08.647436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:45447)
0923 22:08:08.647627 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:08.647693 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":55,"threads_started":1}
W20260923 22:08:08.647781 9191 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:34767 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
W20260923 22:08:08.648005 9327 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:34767 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
I20260923 22:08:08.648540 9396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:08.648308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.193:53147)
0923 22:08:08.648422 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:08.648487 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":45,"threads_started":1}
W20260923 22:08:08.648742 9054 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:34767 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
I20260923 22:08:09.150269 9397 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd775ca101fd4cc2ad5778cbff713bcf)
I20260923 22:08:09.150403 9397 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } }
I20260923 22:08:09.150668 9397 leader_election.cc:288] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767), 02c291ffca5f4e84a28d99b3947d4c07 (127.1.244.252:38891)
I20260923 22:08:09.150939 9398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd775ca101fd4cc2ad5778cbff713bcf)
I20260923 22:08:09.151062 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.150822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:35952)
0923 22:08:09.150948 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.151005 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260923 22:08:09.151086 9398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } }
W20260923 22:08:09.151194 8674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
I20260923 22:08:09.151518 9398 leader_election.cc:288] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767), ae670858be364ed6bc4c88d32a1eced2 (127.1.244.253:41969)
I20260923 22:08:09.151702 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.151419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:35958)
0923 22:08:09.151495 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.151566 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61,"thread_start_us":42,"threads_started":1}
I20260923 22:08:09.151839 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.151517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:35968)
0923 22:08:09.151769 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.151801 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235}
W20260923 22:08:09.151927 8674 leader_election.cc:334] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767): Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
W20260923 22:08:09.151924 8860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
I20260923 22:08:09.152345 9404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.152248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:35972)
0923 22:08:09.152284 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.152315 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 22:08:09.152428 8860 leader_election.cc:334] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767): Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
I20260923 22:08:09.154899 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.150970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37212 (local address 127.1.244.252:38891)
0923 22:08:09.151245 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:09.151247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:09.151260 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:08:09.151592 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:09.151593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:09.151604 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:09.151714 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:09.151872 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.152647 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.153967 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.154546 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.154619 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.154631 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:09.154781 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:08:09.154831 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":54,"threads_started":1}
I20260923 22:08:09.154922 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.150833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:38891 (local address 127.0.0.1:37212)
0923 22:08:09.151053 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.151063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:09.151400 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:09.151723 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:09.151724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:09.151727 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:09.151789 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.151792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.152724 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.152725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.153826 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.153829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.154625 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.154626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.154701 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.154706 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:08:09.154789 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:08:09.154840 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":58,"threads_started":1}
I20260923 22:08:09.155086 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.151634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41969 (local address 127.0.0.1:52240)
0923 22:08:09.151841 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.151852 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:09.152028 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:09.152260 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:09.152261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:09.152264 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:09.152298 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.152299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.153129 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.153130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.154082 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.154086 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.154844 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.154845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.154922 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.154926 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:09.154982 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:08:09.155026 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":46,"threads_started":1}
I20260923 22:08:09.155295 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.151756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.1.244.253:41969)
0923 22:08:09.152147 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:09.152149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:09.152156 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:09.152177 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:09.152178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:09.152188 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:09.152254 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:09.152361 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.153047 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.154148 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.154736 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.154842 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.154853 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:09.155191 (+ 338us) server_negotiation.cc:299] Negotiation successful
0923 22:08:09.155232 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":83,"threads_started":1}
I20260923 22:08:09.155303 8877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ae670858be364ed6bc4c88d32a1eced2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "02c291ffca5f4e84a28d99b3947d4c07" is_pre_election: true
I20260923 22:08:09.155414 8877 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae670858be364ed6bc4c88d32a1eced2 in term 1.
I20260923 22:08:09.155458 8698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02c291ffca5f4e84a28d99b3947d4c07" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ae670858be364ed6bc4c88d32a1eced2" is_pre_election: true
I20260923 22:08:09.155591 8698 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c291ffca5f4e84a28d99b3947d4c07 in term 1.
I20260923 22:08:09.155619 8672 leader_election.cc:302] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c291ffca5f4e84a28d99b3947d4c07, ae670858be364ed6bc4c88d32a1eced2; no voters: bd775ca101fd4cc2ad5778cbff713bcf
I20260923 22:08:09.155700 9397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 22:08:09.155740 9397 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bd775ca101fd4cc2ad5778cbff713bcf)
I20260923 22:08:09.155762 9397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:08:09.155802 8860 leader_election.cc:302] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c291ffca5f4e84a28d99b3947d4c07, ae670858be364ed6bc4c88d32a1eced2; no voters: bd775ca101fd4cc2ad5778cbff713bcf
I20260923 22:08:09.155920 9398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 22:08:09.155980 9398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bd775ca101fd4cc2ad5778cbff713bcf)
I20260923 22:08:09.156013 9398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:08:09.156529 9397 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } }
I20260923 22:08:09.156665 9397 leader_election.cc:288] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [CANDIDATE]: Term 2 election: Requested vote from peers bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767), 02c291ffca5f4e84a28d99b3947d4c07 (127.1.244.252:38891)
I20260923 22:08:09.156687 9398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } }
I20260923 22:08:09.156816 9398 leader_election.cc:288] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 2 election: Requested vote from peers bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767), ae670858be364ed6bc4c88d32a1eced2 (127.1.244.253:41969)
I20260923 22:08:09.156823 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.156739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:35986)
0923 22:08:09.156769 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.156796 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:08:09.156987 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.156902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:36000)
0923 22:08:09.156940 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.156967 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260923 22:08:09.157023 8877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ae670858be364ed6bc4c88d32a1eced2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "02c291ffca5f4e84a28d99b3947d4c07"
I20260923 22:08:09.157106 8877 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ae670858be364ed6bc4c88d32a1eced2 in current term 2: Already voted for candidate 02c291ffca5f4e84a28d99b3947d4c07 in this term.
I20260923 22:08:09.157124 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.157057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:36006)
0923 22:08:09.157086 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.157107 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260923 22:08:09.157250 8674 leader_election.cc:334] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767): Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
I20260923 22:08:09.157250 8698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02c291ffca5f4e84a28d99b3947d4c07" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ae670858be364ed6bc4c88d32a1eced2"
I20260923 22:08:09.157325 8672 leader_election.cc:302] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae670858be364ed6bc4c88d32a1eced2; no voters: 02c291ffca5f4e84a28d99b3947d4c07, bd775ca101fd4cc2ad5778cbff713bcf
I20260923 22:08:09.157367 8698 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c291ffca5f4e84a28d99b3947d4c07 in current term 2: Already voted for candidate ae670858be364ed6bc4c88d32a1eced2 in this term.
I20260923 22:08:09.157420 9397 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 22:08:09.157684 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.157582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:36018)
0923 22:08:09.157633 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.157658 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:08:09.157742 8860 leader_election.cc:334] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767): Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
I20260923 22:08:09.157771 8860 leader_election.cc:302] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c291ffca5f4e84a28d99b3947d4c07; no voters: ae670858be364ed6bc4c88d32a1eced2, bd775ca101fd4cc2ad5778cbff713bcf
I20260923 22:08:09.157842 9398 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 22:08:09.648339 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.648082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.194:55095)
0923 22:08:09.648232 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.648280 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260923 22:08:09.648577 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.648269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:58115)
0923 22:08:09.648410 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.648444 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":63,"threads_started":1}
I20260923 22:08:09.649240 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.649060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.193:41295)
0923 22:08:09.649156 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.649191 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":51,"threads_started":1}
I20260923 22:08:10.649082 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:10.648853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.194:50747)
0923 22:08:10.648976 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:10.649022 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260923 22:08:10.649197 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:10.648986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:44315)
0923 22:08:10.649090 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:10.649128 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
I20260923 22:08:10.649865 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:10.649689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.193:46515)
0923 22:08:10.649786 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:10.649819 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":45,"threads_started":1}
I20260923 22:08:11.031713 9413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:11.031807 9413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } }
I20260923 22:08:11.031985 9413 leader_election.cc:288] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767), ae670858be364ed6bc4c88d32a1eced2 (127.1.244.253:41969)
I20260923 22:08:11.032346 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.032147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:36246)
0923 22:08:11.032256 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.032298 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":65,"threads_started":1}
I20260923 22:08:11.032414 8698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02c291ffca5f4e84a28d99b3947d4c07" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ae670858be364ed6bc4c88d32a1eced2" is_pre_election: true
I20260923 22:08:11.032542 8698 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c291ffca5f4e84a28d99b3947d4c07 in term 2.
I20260923 22:08:11.032755 8860 leader_election.cc:302] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c291ffca5f4e84a28d99b3947d4c07, ae670858be364ed6bc4c88d32a1eced2; no voters:
I20260923 22:08:11.032816 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.032689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:36256)
0923 22:08:11.032742 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.032769 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260923 22:08:11.032851 8860 leader_election.cc:334] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767): Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
I20260923 22:08:11.032893 9413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260923 22:08:11.032944 9413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:11.032976 9413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:08:11.033692 9413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } }
I20260923 22:08:11.033835 9413 leader_election.cc:288] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 3 election: Requested vote from peers bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767), ae670858be364ed6bc4c88d32a1eced2 (127.1.244.253:41969)
I20260923 22:08:11.034022 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.033930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:36272)
0923 22:08:11.033962 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.033992 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:08:11.034159 8698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02c291ffca5f4e84a28d99b3947d4c07" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ae670858be364ed6bc4c88d32a1eced2"
I20260923 22:08:11.034266 8698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:08:11.034358 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.034236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:36278)
0923 22:08:11.034280 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.034305 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 22:08:11.034471 8860 leader_election.cc:334] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767): Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
I20260923 22:08:11.034976 8698 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c291ffca5f4e84a28d99b3947d4c07 in term 3.
I20260923 22:08:11.035169 8860 leader_election.cc:302] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c291ffca5f4e84a28d99b3947d4c07, ae670858be364ed6bc4c88d32a1eced2; no voters: bd775ca101fd4cc2ad5778cbff713bcf
I20260923 22:08:11.035295 9413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:08:11.035418 9413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [term 3 LEADER]: Becoming Leader. State: Replica: 02c291ffca5f4e84a28d99b3947d4c07, State: Running, Role: LEADER
I20260923 22:08:11.035486 9413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } }
I20260923 22:08:11.035876 9416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02c291ffca5f4e84a28d99b3947d4c07. Latest consensus state: current_term: 3 leader_uuid: "02c291ffca5f4e84a28d99b3947d4c07" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:11.035957 9416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:11.036123 9418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:11.036562 9418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:11.036657 9418 catalog_manager.cc:1259] Loaded cluster ID: 3f507a557c87407892d6df79cef3c379
I20260923 22:08:11.036689 9418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:11.036922 9418 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:11.037021 9418 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Loaded TSK: 0
I20260923 22:08:11.037169 9418 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:11.078161 8873 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:11.079882 8873 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:11.080868 8698 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [term 3 FOLLOWER]: Refusing update from remote peer 02c291ffca5f4e84a28d99b3947d4c07: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260923 22:08:11.080897 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.080778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:36306)
0923 22:08:11.080838 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.080869 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:08:11.081094 9416 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260923 22:08:11.081235 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.081138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.0.0.1:36318)
0923 22:08:11.081186 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.081215 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:08:11.081327 8860 consensus_peers.cc:487] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 -> Peer bd775ca101fd4cc2ad5778cbff713bcf (127.1.244.254:34767): Couldn't send request to peer bd775ca101fd4cc2ad5778cbff713bcf. Status: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 22:08:11.081688 9420 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02c291ffca5f4e84a28d99b3947d4c07. Latest consensus state: current_term: 3 leader_uuid: "02c291ffca5f4e84a28d99b3947d4c07" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:11.081782 9420 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:11.082330 9416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "02c291ffca5f4e84a28d99b3947d4c07" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:11.082415 9416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:11.082453 9420 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "02c291ffca5f4e84a28d99b3947d4c07" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd775ca101fd4cc2ad5778cbff713bcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34767 } } peers { permanent_uuid: "ae670858be364ed6bc4c88d32a1eced2" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41969 } attrs { promote: false } } peers { permanent_uuid: "02c291ffca5f4e84a28d99b3947d4c07" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 38891 } attrs { promote: false } } }
I20260923 22:08:11.082549 9420 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae670858be364ed6bc4c88d32a1eced2 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:11.087471 9427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.084345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:39369 (local address 127.0.0.1:58618)
0923 22:08:11.084480 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.084492 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:11.084553 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:11.084739 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:11.084739 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:11.084741 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:11.084772 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.084773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.085684 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.085685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.086489 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.086490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.087228 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.087229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.087289 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.087293 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:11.087346 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:08:11.087385 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":80,"thread_start_us":78,"threads_started":1}
I20260923 22:08:11.087476 9428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.084398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.1.244.194:39369)
0923 22:08:11.084492 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.084494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.084609 (+ 115us) server_negotiation.cc:407] Connection header received
0923 22:08:11.084633 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.084634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.084645 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.084698 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:11.084852 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.085588 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.086671 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.087148 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.087220 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.087231 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.087366 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.087406 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":44,"threads_started":1}
I20260923 22:08:11.087903 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.084511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42541 (local address 127.0.0.1:33382)
0923 22:08:11.084710 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.084769 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:11.084809 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:11.085023 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:11.085025 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:11.085026 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:11.085073 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.085074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.085712 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.085713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.086864 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.086867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.087665 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.087666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.087734 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.087738 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:11.087790 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:08:11.087835 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":34,"thread_start_us":59,"threads_started":1}
I20260923 22:08:11.088110 9430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.084574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33382 (local address 127.1.244.195:42541)
0923 22:08:11.084899 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.084901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.084907 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:08:11.084932 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.084932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.084940 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.085022 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:11.085134 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.085608 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.086963 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.087528 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.087661 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.087675 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.087991 (+ 316us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.088031 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":53,"threads_started":1}
I20260923 22:08:11.088106 9124 tablet_service.cc:1467] Processing CreateTablet for tablet 1b3dc2978cb8456e90c500d275e82133 (DEFAULT_TABLE table=test-table [id=a2f65d02f41945f692e7b8a9e26aea36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:11.088227 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.084236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:33885 (local address 127.0.0.1:47676)
0923 22:08:11.084286 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.084296 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:11.084345 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:11.084684 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:11.084685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:11.084686 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:11.084742 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.084743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.085676 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.085677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.086969 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.086971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.087805 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.087806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.088059 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.088064 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:08:11.088112 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:08:11.088158 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":38}
I20260923 22:08:11.088236 9124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b3dc2978cb8456e90c500d275e82133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:11.088248 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.084305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.1.244.193:33885)
0923 22:08:11.084543 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.084545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.084553 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:11.084579 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.084580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.084591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.084805 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:11.084845 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.085589 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.087059 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.087722 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.087799 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.087811 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.088120 (+ 309us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.088174 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":50,"threads_started":1}
I20260923 22:08:11.088280 9260 tablet_service.cc:1467] Processing CreateTablet for tablet 1b3dc2978cb8456e90c500d275e82133 (DEFAULT_TABLE table=test-table [id=a2f65d02f41945f692e7b8a9e26aea36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:11.088398 9260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b3dc2978cb8456e90c500d275e82133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:11.088425 8987 tablet_service.cc:1467] Processing CreateTablet for tablet 1b3dc2978cb8456e90c500d275e82133 (DEFAULT_TABLE table=test-table [id=a2f65d02f41945f692e7b8a9e26aea36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:11.088524 8987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b3dc2978cb8456e90c500d275e82133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:11.089535 9431 tablet_bootstrap.cc:492] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: Bootstrap starting.
I20260923 22:08:11.089923 9431 tablet_bootstrap.cc:654] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:11.089967 9433 tablet_bootstrap.cc:492] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98: Bootstrap starting.
I20260923 22:08:11.090276 9432 tablet_bootstrap.cc:492] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: Bootstrap starting.
I20260923 22:08:11.090313 9433 tablet_bootstrap.cc:654] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:11.090488 9431 tablet_bootstrap.cc:492] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: No bootstrap required, opened a new log
I20260923 22:08:11.090560 9431 ts_tablet_manager.cc:1397] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:11.090643 9432 tablet_bootstrap.cc:654] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:11.090725 9431 raft_consensus.cc:348] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } }
I20260923 22:08:11.090802 9431 raft_consensus.cc:374] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:11.090818 9431 raft_consensus.cc:729] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58c1112264e4386a4cce33487fe2dc3, State: Initialized, Role: FOLLOWER
I20260923 22:08:11.090889 9431 consensus_queue.cc:260] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } }
I20260923 22:08:11.090993 9433 tablet_bootstrap.cc:492] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98: No bootstrap required, opened a new log
I20260923 22:08:11.091045 9433 ts_tablet_manager.cc:1397] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:11.091081 9431 ts_tablet_manager.cc:1428] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:11.091147 9433 raft_consensus.cc:348] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } }
I20260923 22:08:11.091208 9433 raft_consensus.cc:374] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:11.091233 9433 raft_consensus.cc:729] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec38f76df844757bbcfec9a04b56e98, State: Initialized, Role: FOLLOWER
I20260923 22:08:11.091391 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.091291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:45883)
0923 22:08:11.091333 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.091366 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 22:08:11.091660 9326 heartbeater.cc:503] Master 127.1.244.252:38891 was elected leader, sending a full tablet report...
I20260923 22:08:11.091741 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.091608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:55803)
0923 22:08:11.091668 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.091689 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 22:08:11.091292 9433 consensus_queue.cc:260] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } }
I20260923 22:08:11.092029 9433 ts_tablet_manager.cc:1428] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:11.092350 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.092257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.193:54453)
0923 22:08:11.092292 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.092324 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 22:08:11.092567 9432 tablet_bootstrap.cc:492] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: No bootstrap required, opened a new log
I20260923 22:08:11.092607 9432 ts_tablet_manager.cc:1397] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:11.092787 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.092697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.193:47925)
0923 22:08:11.092729 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.092757 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:08:11.092770 9432 raft_consensus.cc:348] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } }
I20260923 22:08:11.092847 9432 raft_consensus.cc:374] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:11.092869 9432 raft_consensus.cc:729] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5567facef3cf4b4f845f9af30f43580b, State: Initialized, Role: FOLLOWER
I20260923 22:08:11.092926 9432 consensus_queue.cc:260] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } }
I20260923 22:08:11.093034 9053 heartbeater.cc:503] Master 127.1.244.252:38891 was elected leader, sending a full tablet report...
I20260923 22:08:11.093137 9432 ts_tablet_manager.cc:1428] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:11.093393 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.093283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.194:60979)
0923 22:08:11.093323 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.093365 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 22:08:11.093650 9190 heartbeater.cc:503] Master 127.1.244.252:38891 was elected leader, sending a full tablet report...
I20260923 22:08:11.093770 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.093588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.194:50491)
0923 22:08:11.093711 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.093738 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260923 22:08:11.489233 9437 raft_consensus.cc:482] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:11.489329 9437 raft_consensus.cc:504] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } }
I20260923 22:08:11.489674 9437 leader_election.cc:288] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec38f76df844757bbcfec9a04b56e98 (127.1.244.193:33885), 5567facef3cf4b4f845f9af30f43580b (127.1.244.194:39369)
I20260923 22:08:11.492583 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.489981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:59399 (local address 127.1.244.193:33885)
0923 22:08:11.490050 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.490052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.490064 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:08:11.490098 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.490099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.490109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.490181 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:11.490340 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.490837 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.491857 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.492229 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.492317 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.492327 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.492478 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.492516 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:08:11.492585 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.489908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:33885 (local address 127.1.244.195:59399)
0923 22:08:11.489970 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.489984 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:11.490038 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:11.490201 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:11.490202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:11.490205 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:11.490267 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.490269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.490933 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.490934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.491768 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.491770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.492325 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.492326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.492403 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.492406 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:11.492472 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:08:11.492512 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":2}
I20260923 22:08:11.492861 9428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.490113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:50843 (local address 127.1.244.194:39369)
0923 22:08:11.490181 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.490183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.490294 (+ 111us) server_negotiation.cc:407] Connection header received
0923 22:08:11.490370 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.490371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.490387 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.490467 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:11.490602 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.491052 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.492037 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.492385 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.492484 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.492493 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.492764 (+ 271us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.492804 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:08:11.492851 9007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b3dc2978cb8456e90c500d275e82133" candidate_uuid: "e58c1112264e4386a4cce33487fe2dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec38f76df844757bbcfec9a04b56e98" is_pre_election: true
I20260923 22:08:11.492996 9007 raft_consensus.cc:2454] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e58c1112264e4386a4cce33487fe2dc3 in term 0.
I20260923 22:08:11.492776 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.490032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:39369 (local address 127.1.244.195:50843)
0923 22:08:11.490226 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.490241 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:11.490305 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:11.490479 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:11.490480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:11.490481 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:11.490526 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.490528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.491136 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.491137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.491936 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.491938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.492484 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.492485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.492551 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.492554 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:11.492628 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:08:11.492675 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":1,"thread_start_us":136,"threads_started":1}
I20260923 22:08:11.493198 9216 leader_election.cc:302] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec38f76df844757bbcfec9a04b56e98, e58c1112264e4386a4cce33487fe2dc3; no voters:
I20260923 22:08:11.493369 9437 raft_consensus.cc:2781] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:11.493413 9437 raft_consensus.cc:482] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:11.493435 9437 raft_consensus.cc:3037] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:11.493484 9144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b3dc2978cb8456e90c500d275e82133" candidate_uuid: "e58c1112264e4386a4cce33487fe2dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5567facef3cf4b4f845f9af30f43580b" is_pre_election: true
I20260923 22:08:11.493570 9144 raft_consensus.cc:2454] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e58c1112264e4386a4cce33487fe2dc3 in term 0.
I20260923 22:08:11.494235 9437 raft_consensus.cc:504] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } }
I20260923 22:08:11.494375 9437 leader_election.cc:288] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [CANDIDATE]: Term 1 election: Requested vote from peers 0ec38f76df844757bbcfec9a04b56e98 (127.1.244.193:33885), 5567facef3cf4b4f845f9af30f43580b (127.1.244.194:39369)
I20260923 22:08:11.494537 9007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b3dc2978cb8456e90c500d275e82133" candidate_uuid: "e58c1112264e4386a4cce33487fe2dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec38f76df844757bbcfec9a04b56e98"
I20260923 22:08:11.494637 9007 raft_consensus.cc:3037] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:11.495280 9007 raft_consensus.cc:2454] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e58c1112264e4386a4cce33487fe2dc3 in term 1.
I20260923 22:08:11.495453 9216 leader_election.cc:302] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec38f76df844757bbcfec9a04b56e98, e58c1112264e4386a4cce33487fe2dc3; no voters:
I20260923 22:08:11.495424 9144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b3dc2978cb8456e90c500d275e82133" candidate_uuid: "e58c1112264e4386a4cce33487fe2dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5567facef3cf4b4f845f9af30f43580b"
I20260923 22:08:11.495507 9144 raft_consensus.cc:3037] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:11.495577 9437 raft_consensus.cc:2781] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:11.495756 9437 raft_consensus.cc:686] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 1 LEADER]: Becoming Leader. State: Replica: e58c1112264e4386a4cce33487fe2dc3, State: Running, Role: LEADER
I20260923 22:08:11.495838 9437 consensus_queue.cc:237] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } }
I20260923 22:08:11.496306 9144 raft_consensus.cc:2454] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e58c1112264e4386a4cce33487fe2dc3 in term 1.
I20260923 22:08:11.496644 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.496550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:48569)
0923 22:08:11.496581 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.496614 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:08:11.496625 8873 catalog_manager.cc:5495] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e58c1112264e4386a4cce33487fe2dc3 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "e58c1112264e4386a4cce33487fe2dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e58c1112264e4386a4cce33487fe2dc3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42541 } health_report { overall_health: HEALTHY } } }
I20260923 22:08:11.527295 9430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.525110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.1.244.195:42541)
0923 22:08:11.525177 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.525178 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.525186 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:11.525233 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.525233 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.525252 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.525313 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:11.525436 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.525904 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.526587 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.526765 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.526814 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.526931 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:11.527071 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:11.527128 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.527167 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.527184 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:08:11.528364 9144 raft_consensus.cc:1261] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [term 1 FOLLOWER]: Refusing update from remote peer e58c1112264e4386a4cce33487fe2dc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:11.528443 9007 raft_consensus.cc:1261] T 1b3dc2978cb8456e90c500d275e82133 P 0ec38f76df844757bbcfec9a04b56e98 [term 1 FOLLOWER]: Refusing update from remote peer e58c1112264e4386a4cce33487fe2dc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:11.528648 9437 consensus_queue.cc:1035] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5567facef3cf4b4f845f9af30f43580b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:11.528767 9441 consensus_queue.cc:1035] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec38f76df844757bbcfec9a04b56e98" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:11.529958 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.529841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.194:33061)
0923 22:08:11.529883 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.529924 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260923 22:08:11.530143 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.529894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.193:37145)
0923 22:08:11.530062 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.530097 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260923 22:08:11.530584 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.530490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:53559)
0923 22:08:11.530530 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.530555 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:08:11.531342 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.531253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.193:35535)
0923 22:08:11.531291 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.531316 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 22:08:11.531749 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.531663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.195:39273)
0923 22:08:11.531698 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.531721 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 22:08:11.532876 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.532794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34767 (local address 127.1.244.194:59355)
0923 22:08:11.532829 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.532851 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:34767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 22:08:11.532900 8874 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:11.532999 8874 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:11.534261 8874 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02c291ffca5f4e84a28d99b3947d4c07: Sending DeleteTablet for 3 replicas of tablet 1b3dc2978cb8456e90c500d275e82133
I20260923 22:08:11.534600 9260 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b3dc2978cb8456e90c500d275e82133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:11 UTC) from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:33382
I20260923 22:08:11.534650 8987 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b3dc2978cb8456e90c500d275e82133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:11 UTC) from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:47676
I20260923 22:08:11.534787 9454 tablet_replica.cc:330] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: stopping tablet replica
I20260923 22:08:11.534840 9454 raft_consensus.cc:2229] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:11.534862 9124 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b3dc2978cb8456e90c500d275e82133 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:11 UTC) from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:58618
I20260923 22:08:11.534920 9454 raft_consensus.cc:2258] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:11.535130 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 8835
I20260923 22:08:11.536906 9454 ts_tablet_manager.cc:1905] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:11.538897 9454 ts_tablet_manager.cc:1918] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:11.538962 9454 log.cc:1196] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/1b3dc2978cb8456e90c500d275e82133
I20260923 22:08:11.539188 9454 ts_tablet_manager.cc:1939] T 1b3dc2978cb8456e90c500d275e82133 P e58c1112264e4386a4cce33487fe2dc3: Deleting consensus metadata
I20260923 22:08:11.539598 8861 catalog_manager.cc:4841] TS e58c1112264e4386a4cce33487fe2dc3 (127.1.244.195:42541): tablet 1b3dc2978cb8456e90c500d275e82133 (table test-table [id=a2f65d02f41945f692e7b8a9e26aea36]) successfully deleted
I20260923 22:08:11.539629 9456 tablet_replica.cc:330] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: stopping tablet replica
I20260923 22:08:11.539685 9456 raft_consensus.cc:2229] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:11.539726 9456 raft_consensus.cc:2258] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:11.540028 9456 ts_tablet_manager.cc:1905] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:11.541502 9456 ts_tablet_manager.cc:1918] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:11.541566 9456 log.cc:1196] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/1b3dc2978cb8456e90c500d275e82133
I20260923 22:08:11.541792 9456 ts_tablet_manager.cc:1939] T 1b3dc2978cb8456e90c500d275e82133 P 5567facef3cf4b4f845f9af30f43580b: Deleting consensus metadata
I20260923 22:08:11.542094 8861 catalog_manager.cc:4841] TS 5567facef3cf4b4f845f9af30f43580b (127.1.244.194:39369): tablet 1b3dc2978cb8456e90c500d275e82133 (table test-table [id=a2f65d02f41945f692e7b8a9e26aea36]) successfully deleted
I20260923 22:08:11.543088 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.542957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:33885 (local address 127.0.0.1:47680)
0923 22:08:11.543021 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.543052 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:33885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 22:08:11.543232 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 9061
W20260923 22:08:11.543220 8861 catalog_manager.cc:4584] TS 0ec38f76df844757bbcfec9a04b56e98 (127.1.244.193:33885): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b3dc2978cb8456e90c500d275e82133: Network error: Client connection negotiation failed: client connection to 127.1.244.193:33885: connect: Connection refused (error 111)
I20260923 22:08:11.549525 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 9197
I20260923 22:08:11.554616 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 8554
I20260923 22:08:11.559224 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 8742
2026-09-23T22:08:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7102 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:11 dist-test-slave-b5jz krb5kdc[9459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:11 dist-test-slave-b5jz krb5kdc[9459](info): set up 2 sockets
Sep 23 22:08:11 dist-test-slave-b5jz krb5kdc[9459](info): commencing operation
krb5kdc: starting...
W20260923 22:08:12.596098 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:08:12 dist-test-slave-b5jz krb5kdc[9459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.254.keytab.
Entry for principal oryx/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.254.keytab.
I20260923 22:08:12.712639 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37371
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.254.keytab
--principal=oryx/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:32971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:37371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:12.791842 9472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:12.792043 9472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:12.792065 9472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:12.793272 9472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:12.793318 9472 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:08:12.793331 9472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:12.793344 9472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:12.793357 9472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:12.793370 9472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:12.794811 9472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:32971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:37371
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.254.keytab
--principal=oryx/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:12.794999 9472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:12.795161 9472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:12.800061 9480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:12.800042 9477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:12.800067 9478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:12.800102 9472 server_base.cc:1034] running on GCE node
I20260923 22:08:12.800549 9472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:12.800833 9472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:12.801994 9472 hybrid_clock.cc:648] HybridClock initialized: now 1790201292801982 us; error 44 us; skew 500 ppm
Sep 23 22:08:12 dist-test-slave-b5jz krb5kdc[9459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:12.809245 9472 init.cc:377] Logged in from keytab as oryx/127.1.244.254@KRBTEST.COM (short username oryx)
I20260923 22:08:12.809679 9472 webserver.cc:460] Webserver started at http://127.1.244.254:35609/ using document root <none> and password file <none>
I20260923 22:08:12.809903 9472 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:12.809988 9472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:12.810138 9472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:12.811187 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "59a05404acc64309a8525f5865050e02"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-b5jz"
server_key: "6ab69b9d97dc737ae9722342db1c43c9"
server_key_iv: "0979ac9ddf40e937851b9506247fb2a5"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.811601 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "59a05404acc64309a8525f5865050e02"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-b5jz"
server_key: "6ab69b9d97dc737ae9722342db1c43c9"
server_key_iv: "0979ac9ddf40e937851b9506247fb2a5"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.812932 9472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:12.813666 9487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.813838 9472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:12.813948 9472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "59a05404acc64309a8525f5865050e02"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-b5jz"
server_key: "6ab69b9d97dc737ae9722342db1c43c9"
server_key_iv: "0979ac9ddf40e937851b9506247fb2a5"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.814062 9472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:12.827363 9472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:12.829245 9472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:12.829417 9472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:12.833972 9472 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:37371
I20260923 22:08:12.834043 9549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:37371 every 8 connection(s)
I20260923 22:08:12.834394 9472 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:12.835108 9550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:12.837190 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 9472
I20260923 22:08:12.837318 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:12.837518 2003 external_mini_cluster.cc:1428] Setting key 409cb1b7bdf65950c3580968f13669e3
I20260923 22:08:12.837554 9550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02: Bootstrap starting.
I20260923 22:08:12.838289 9550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:12.838605 9550 log.cc:824] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:12.839301 9550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02: No bootstrap required, opened a new log
I20260923 22:08:12.841192 9550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a05404acc64309a8525f5865050e02" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37371 } }
I20260923 22:08:12.841359 9550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:12.841408 9550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59a05404acc64309a8525f5865050e02, State: Initialized, Role: FOLLOWER
I20260923 22:08:12.841537 9550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a05404acc64309a8525f5865050e02" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37371 } }
I20260923 22:08:12.841634 9550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:12.841678 9550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:12.841727 9550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:12.842238 9550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a05404acc64309a8525f5865050e02" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37371 } }
I20260923 22:08:12.842357 9550 leader_election.cc:302] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59a05404acc64309a8525f5865050e02; no voters:
I20260923 22:08:12.842510 9550 leader_election.cc:288] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:12.842543 9555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:12.842725 9555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [term 1 LEADER]: Becoming Leader. State: Replica: 59a05404acc64309a8525f5865050e02, State: Running, Role: LEADER
I20260923 22:08:12.842785 9550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:12.842810 9555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a05404acc64309a8525f5865050e02" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37371 } }
I20260923 22:08:12.843092 9556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59a05404acc64309a8525f5865050e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a05404acc64309a8525f5865050e02" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37371 } } }
I20260923 22:08:12.843076 9555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59a05404acc64309a8525f5865050e02. Latest consensus state: current_term: 1 leader_uuid: "59a05404acc64309a8525f5865050e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a05404acc64309a8525f5865050e02" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37371 } } }
I20260923 22:08:12.843163 9556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:12.843173 9555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:12.843350 9560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:12.843899 9560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:08:12 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:12.845662 9560 catalog_manager.cc:1347] Generated new cluster ID: 7c855c2ae02544d2bc8d7cf231ba392d
I20260923 22:08:12.845712 9560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:12.851140 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:12.838033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55424 (local address 127.1.244.254:37371)
0923 22:08:12.838212 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:12.838220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:12.838245 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:08:12.838728 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:12.838744 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:12.838795 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:12.838951 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:12.839657 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.840526 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.841278 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.841895 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.842030 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:12.845264 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:12.845269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:12.845270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:12.845283 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:12.849301 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.849693 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.849696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.849696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.849794 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.849980 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.849981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.849981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.850269 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:12.850405 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:12.850811 (+ 406us) server_negotiation.cc:299] Negotiation successful
0923 22:08:12.850943 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260923 22:08:12.859299 9560 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:12.860035 9560 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:12.863312 9560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02: Generated new TSK 0
I20260923 22:08:12.863472 9560 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
I20260923 22:08:12.895934 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37371
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:32971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:13.006558 9576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:13.006755 9576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:13.006779 9576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:13.007973 9576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:13.008013 9576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:13.008034 9576 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:08:13.008052 9576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:13.009546 9576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:32971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:13.009788 9576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:13.010051 9576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:13.014990 9581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:13.015023 9582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.015276 9576 server_base.cc:1034] running on GCE node
W20260923 22:08:13.015015 9584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.015615 9576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:13.015875 9576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:13.017038 9576 hybrid_clock.cc:648] HybridClock initialized: now 1790201293017025 us; error 30 us; skew 500 ppm
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201293, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:13.019737 9576 init.cc:377] Logged in from keytab as oryx/127.1.244.193@KRBTEST.COM (short username oryx)
I20260923 22:08:13.020202 9576 webserver.cc:460] Webserver started at http://127.1.244.193:38251/ using document root <none> and password file <none>
I20260923 22:08:13.020445 9576 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:13.020498 9576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:13.020627 9576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:13.021973 9576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "7d3989ca76c34afc80f4bd1a98106943"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-b5jz"
server_key: "3fe93eb7157a2dd929830ed997b1f26b"
server_key_iv: "528112f280f337b04e98e75cd959c289"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.022392 9576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "7d3989ca76c34afc80f4bd1a98106943"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-b5jz"
server_key: "3fe93eb7157a2dd929830ed997b1f26b"
server_key_iv: "528112f280f337b04e98e75cd959c289"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.024006 9576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260923 22:08:13.024904 9591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.025110 9576 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:13.025213 9576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "7d3989ca76c34afc80f4bd1a98106943"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-b5jz"
server_key: "3fe93eb7157a2dd929830ed997b1f26b"
server_key_iv: "528112f280f337b04e98e75cd959c289"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.025288 9576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:13.033051 9576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:13.035276 9576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:13.035439 9576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:13.035928 9576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:13.035971 9576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.035997 9576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:13.036044 9576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.042446 9576 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:41851
I20260923 22:08:13.042850 9576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201293, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:13.043519 9706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:41851 every 8 connection(s)
I20260923 22:08:13.046949 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.036440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:42803 (local address 127.1.244.254:37371)
0923 22:08:13.036487 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.036489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.036712 (+ 223us) server_negotiation.cc:407] Connection header received
0923 22:08:13.037246 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.037247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.037259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.037302 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.038102 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.038938 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.040664 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.040859 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.040919 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.044017 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.044018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.044018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:13.044026 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.045550 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.045963 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.045964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.045965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.046032 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.046293 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.046294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.046294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.046414 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.046488 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.046763 (+ 275us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.046858 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:13.047108 9600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.036435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37371 (local address 127.1.244.193:42803)
0923 22:08:13.036596 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.036629 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.037121 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.037377 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.037382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.037731 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.038010 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.038017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.039074 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.039076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.040567 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.040570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.040953 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.040955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.041171 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.041757 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:13.041768 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.043921 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.045654 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.045661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.045663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.045884 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.046099 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.046100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.046100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.046210 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.046489 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.046496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.046630 (+ 134us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.046735 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.046887 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260923 22:08:13.047679 9707 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37371
I20260923 22:08:13.047768 9707 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:13.047962 9707 heartbeater.cc:511] Master 127.1.244.254:37371 requested a full tablet report, sending...
I20260923 22:08:13.048494 9503 ts_manager.cc:194] Registered new tserver with Master: 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851)
I20260923 22:08:13.049180 9503 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:42803
I20260923 22:08:13.051898 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 9576
I20260923 22:08:13.051986 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:13.052210 2003 external_mini_cluster.cc:1428] Setting key 15c3149d3f5007f303a924f3bd9bd841
I20260923 22:08:13.054814 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.048701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37371 (local address 127.1.244.193:53931)
0923 22:08:13.048817 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.048827 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.048879 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.049470 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.049471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.049673 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.049730 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.049731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.050356 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.050357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.050859 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.050860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.051249 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.051250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.051325 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.051755 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:13.051759 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.052107 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.053777 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.053778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.053779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.054010 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.054200 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.054201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.054202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.054276 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.054572 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.054573 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.054608 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.054659 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.054731 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":68,"threads_started":1}
I20260923 22:08:13.054890 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.048765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53931 (local address 127.1.244.254:37371)
0923 22:08:13.049370 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.049371 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.049379 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:13.049404 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.049404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.049412 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.049459 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.049811 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.050262 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.050960 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.051160 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.051246 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.052187 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.052188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.052188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:13.052195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.053688 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.054089 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.054089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.054089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.054138 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.054359 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.054360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.054360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.054506 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.054578 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.054699 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.054776 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584}
WARNING: no policy specified for oryx/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
I20260923 22:08:13.078190 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37371
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:32971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:13.157789 9716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:13.157995 9716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:13.158020 9716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:13.159101 9716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:13.159135 9716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:13.159155 9716 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:08:13.159173 9716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:13.160640 9716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:32971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:13.160827 9716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:13.161003 9716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:13.166285 9716 server_base.cc:1034] running on GCE node
W20260923 22:08:13.166268 9721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:13.166263 9722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:13.166249 9724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.166640 9716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:13.166884 9716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:13.168042 9716 hybrid_clock.cc:648] HybridClock initialized: now 1790201293168031 us; error 45 us; skew 500 ppm
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201293, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:13.170481 9716 init.cc:377] Logged in from keytab as oryx/127.1.244.194@KRBTEST.COM (short username oryx)
I20260923 22:08:13.170962 9716 webserver.cc:460] Webserver started at http://127.1.244.194:39133/ using document root <none> and password file <none>
I20260923 22:08:13.171195 9716 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:13.171264 9716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:13.171416 9716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:13.172531 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "230612f73ece4f33aadd80548bfbaa84"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-b5jz"
server_key: "1693d97d18d29d91bb9b9c22cc547b4a"
server_key_iv: "1d2a3fd5cba22c49f6e81108d101cc5a"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.172894 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "230612f73ece4f33aadd80548bfbaa84"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-b5jz"
server_key: "1693d97d18d29d91bb9b9c22cc547b4a"
server_key_iv: "1d2a3fd5cba22c49f6e81108d101cc5a"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.174201 9716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:13.174937 9731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.175112 9716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:13.175210 9716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "230612f73ece4f33aadd80548bfbaa84"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-b5jz"
server_key: "1693d97d18d29d91bb9b9c22cc547b4a"
server_key_iv: "1d2a3fd5cba22c49f6e81108d101cc5a"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.175300 9716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:13.196573 9716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:13.198422 9716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:13.198596 9716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:13.199211 9716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:13.199260 9716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.199296 9716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:13.199321 9716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201293, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:13.206678 9716 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:38869
I20260923 22:08:13.207018 9716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:13.207185 9846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:38869 every 8 connection(s)
I20260923 22:08:13.209589 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.199674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:36619 (local address 127.1.244.254:37371)
0923 22:08:13.199746 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.199749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.200035 (+ 286us) server_negotiation.cc:407] Connection header received
0923 22:08:13.200766 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.200768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.200786 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.200841 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.201784 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.202355 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.204109 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.204328 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.204425 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.207092 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.207094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.207094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:13.207102 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.208446 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.208804 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.208804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.208805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.208855 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.209063 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.209063 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.209064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.209161 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.209210 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.209450 (+ 240us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.209508 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:08:13.209827 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.199681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37371 (local address 127.1.244.194:36619)
0923 22:08:13.199864 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.199921 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.200640 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.200924 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.200928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.201365 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.201671 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.201679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.202472 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.202473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.204005 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.204009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.204445 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.204446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.204610 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.205109 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:13.205121 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.206990 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.208522 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.208527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.208529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.208738 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.208913 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.208913 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.208914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.209009 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.209215 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.209219 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.209329 (+ 110us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.209434 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.209566 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260923 22:08:13.210310 9847 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37371
I20260923 22:08:13.210405 9847 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:13.210574 9847 heartbeater.cc:511] Master 127.1.244.254:37371 requested a full tablet report, sending...
I20260923 22:08:13.211041 9504 ts_manager.cc:194] Registered new tserver with Master: 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194:38869)
I20260923 22:08:13.211550 9504 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:36619
I20260923 22:08:13.212668 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 9716
I20260923 22:08:13.212785 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:13.212972 2003 external_mini_cluster.cc:1428] Setting key 3cb9f35732f8b7bb91b1b608e67e5160
I20260923 22:08:13.217290 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.211381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:48833 (local address 127.1.244.254:37371)
0923 22:08:13.211712 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.211714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.211723 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:13.211742 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.211743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.211752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.211803 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.212183 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.212718 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.213580 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.213781 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.213863 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.214702 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.214704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.214704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:13.214711 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.216143 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.216545 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.216546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.216546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.216598 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.216818 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.216818 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.216818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.216955 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.217008 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.217168 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.217234 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260923 22:08:13.217425 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.211339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37371 (local address 127.1.244.194:48833)
0923 22:08:13.211468 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.211478 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.211529 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.211820 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.211821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.212022 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.212106 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.212107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.212845 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.212846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.213503 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.213505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.213865 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.213865 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.213929 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.214357 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:13.214361 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.214616 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.216243 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.216244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.216245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.216473 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.216687 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.216688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.216688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.216758 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.217020 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.217021 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.217070 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.217148 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.217270 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for oryx/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
I20260923 22:08:13.238430 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37371
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:32971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:13.318185 9856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:13.318372 9856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:13.318394 9856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:13.319465 9856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:13.319499 9856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:13.319519 9856 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:08:13.319536 9856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:13.321036 9856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:32971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:37371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:13.321214 9856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:13.321372 9856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:13.326267 9862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:13.326272 9861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.326380 9856 server_base.cc:1034] running on GCE node
W20260923 22:08:13.326370 9864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.326704 9856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:13.326969 9856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:13.328131 9856 hybrid_clock.cc:648] HybridClock initialized: now 1790201293328120 us; error 52 us; skew 500 ppm
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201293, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:13.330600 9856 init.cc:377] Logged in from keytab as oryx/127.1.244.195@KRBTEST.COM (short username oryx)
I20260923 22:08:13.331009 9856 webserver.cc:460] Webserver started at http://127.1.244.195:34487/ using document root <none> and password file <none>
I20260923 22:08:13.331243 9856 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:13.331310 9856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:13.331465 9856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:13.332489 9856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "9f491a37f2184bcd83dba05de659aa75"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-b5jz"
server_key: "a44afd45f17397706d5d17695c72b355"
server_key_iv: "231a2cc6e54f01c08e923d1f3a3a6b67"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.332867 9856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "9f491a37f2184bcd83dba05de659aa75"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-b5jz"
server_key: "a44afd45f17397706d5d17695c72b355"
server_key_iv: "231a2cc6e54f01c08e923d1f3a3a6b67"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.334159 9856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:13.334836 9871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.335001 9856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:13.335096 9856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "9f491a37f2184bcd83dba05de659aa75"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-b5jz"
server_key: "a44afd45f17397706d5d17695c72b355"
server_key_iv: "231a2cc6e54f01c08e923d1f3a3a6b67"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.335173 9856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:13.355736 9856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:13.357563 9856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:13.357728 9856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:13.358229 9856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:13.358296 9856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.358340 9856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:13.358369 9856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201293, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:13.365756 9856 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:36567
I20260923 22:08:13.365828 9986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:36567 every 8 connection(s)
I20260923 22:08:13.366200 9856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:13.368075 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.358803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:47663 (local address 127.1.244.254:37371)
0923 22:08:13.358847 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.358850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.359077 (+ 227us) server_negotiation.cc:407] Connection header received
0923 22:08:13.359653 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.359655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.359668 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.359732 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.360368 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.360874 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.361967 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.362291 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.362360 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.365159 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.365161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.365161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:13.365171 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.366844 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.367183 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.367184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.367185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.367239 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.367426 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.367426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.367427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.367526 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.367647 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.367915 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.367990 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:13.368211 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.358776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37371 (local address 127.1.244.195:47663)
0923 22:08:13.358955 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.358989 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.359525 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.359819 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.359823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.360077 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.360266 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.360271 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.360980 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.360981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.361867 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.361870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.362407 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.362408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.362592 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.363200 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:13.363210 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.365056 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.366903 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.366906 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.366908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.367116 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.367281 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.367282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.367282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.367377 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.367638 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.367642 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.367765 (+ 123us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.367874 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.367992 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260923 22:08:13.368693 9987 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37371
I20260923 22:08:13.368775 9987 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:13.368961 9987 heartbeater.cc:511] Master 127.1.244.254:37371 requested a full tablet report, sending...
I20260923 22:08:13.369365 9503 ts_manager.cc:194] Registered new tserver with Master: 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567)
I20260923 22:08:13.369696 9503 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:47663
I20260923 22:08:13.373262 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 9856
I20260923 22:08:13.373349 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:13.373533 2003 external_mini_cluster.cc:1428] Setting key 8e60d76fdb59bd5a47773d437658997f
I20260923 22:08:13.374317 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:08:13.375180 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.369810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:58231 (local address 127.1.244.254:37371)
0923 22:08:13.369870 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.369872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.369999 (+ 127us) server_negotiation.cc:407] Connection header received
0923 22:08:13.370026 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.370027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.370036 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.370105 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.370386 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.370919 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.371589 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.371778 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.371829 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.372562 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.372563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.372563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:13.372569 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.373979 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.374361 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.374362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.374362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.374436 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.374727 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.374728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.374728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.374837 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.374902 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.375064 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.375123 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:08:13.375295 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.369722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37371 (local address 127.1.244.195:58231)
0923 22:08:13.369926 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.369936 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.369992 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.370110 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.370111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.370258 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.370314 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.370315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.371000 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.371001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.371483 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.371484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.371840 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.371841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.371889 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.372208 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:13.372212 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.372489 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.374054 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.374055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.374056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.374282 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.374506 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.374507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.374508 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.374569 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.374910 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.374911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.374957 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.375033 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.375147 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":143,"threads_started":1}
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201293, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:08:13.395654 9553 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:13.390328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55440 (local address 127.1.244.254:37371)
0923 22:08:13.390396 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.390397 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.390406 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:13.390440 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.390441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.390451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.390501 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.391044 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.391538 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.392289 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.392482 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.392535 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.395547 (+ 3012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55440: BlockingRecv error: recv got EOF from 127.0.0.1:55440 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201293, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:13.402900 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.396392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55448 (local address 127.1.244.254:37371)
0923 22:08:13.396462 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.396465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.396546 (+ 81us) server_negotiation.cc:407] Connection header received
0923 22:08:13.396634 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.396635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.396645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.396716 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.397170 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.397642 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.398319 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.398507 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.398590 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.400676 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.400677 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.400677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:13.400685 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.401821 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.402146 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.402148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.402148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.402209 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.402409 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.402410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.402411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.402523 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.402626 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.402724 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.402798 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:08:13.404495 9503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:13.405998 9503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:13.412494 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.408223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:36567 (local address 127.0.0.1:44578)
0923 22:08:13.408450 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.408460 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.408503 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.409303 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.409303 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.409305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.409341 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.409342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.410523 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.410524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.411259 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.411261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.412153 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.412154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.412239 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.412253 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.412356 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.412412 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":80,"threads_started":1}
I20260923 22:08:13.412503 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.408439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.1.244.195:36567)
0923 22:08:13.409144 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.409149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.409158 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:13.409193 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.409195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.409209 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.409287 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.409402 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.410421 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.411357 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.412048 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.412148 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.412229 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.412373 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.412417 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":75,"threads_started":1}
I20260923 22:08:13.412707 10011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.408223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:38869 (local address 127.0.0.1:39938)
0923 22:08:13.408584 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.408591 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.408623 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.408779 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.408782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.408783 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.408831 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.408832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.410120 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.410121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.411374 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.411376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.412460 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.412461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.412522 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.412526 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.412577 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.412626 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":42,"thread_start_us":80,"threads_started":1}
I20260923 22:08:13.412813 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.408191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41851 (local address 127.0.0.1:45700)
0923 22:08:13.408357 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.408396 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.408482 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.409014 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.409015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.409018 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.409066 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.409068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.410040 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.410041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.411618 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.411620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.412615 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.412616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.412668 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.412671 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.412713 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.412752 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":51,"thread_start_us":54,"threads_started":1}
I20260923 22:08:13.412977 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.408456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39938 (local address 127.1.244.194:38869)
0923 22:08:13.408625 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.408629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.408640 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:08:13.408677 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.408678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.408690 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.408758 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.408937 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.410045 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.411467 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.412274 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.412345 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.412421 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.412807 (+ 386us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.412854 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260923 22:08:13.413297 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.408289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.1.244.193:41851)
0923 22:08:13.408854 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.408857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.408864 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:13.408917 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.408919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.408935 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.409017 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.409153 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.409951 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.411708 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.412436 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.412491 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.412559 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.412960 (+ 401us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.413006 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":85,"threads_started":1}
I20260923 22:08:13.413568 9781 tablet_service.cc:1467] Processing CreateTablet for tablet 055ee8d1e31f4f2c88df9462fa383d25 (DEFAULT_TABLE table=test-table [id=7e99fd5c735e4638b4695fb5d0e99c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.413580 9921 tablet_service.cc:1467] Processing CreateTablet for tablet 055ee8d1e31f4f2c88df9462fa383d25 (DEFAULT_TABLE table=test-table [id=7e99fd5c735e4638b4695fb5d0e99c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.413851 9781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 055ee8d1e31f4f2c88df9462fa383d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.413923 9921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 055ee8d1e31f4f2c88df9462fa383d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.416855 10015 tablet_bootstrap.cc:492] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: Bootstrap starting.
I20260923 22:08:13.416873 10016 tablet_bootstrap.cc:492] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: Bootstrap starting.
I20260923 22:08:13.417482 10016 tablet_bootstrap.cc:654] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.417595 10015 tablet_bootstrap.cc:654] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.417584 9641 tablet_service.cc:1467] Processing CreateTablet for tablet 055ee8d1e31f4f2c88df9462fa383d25 (DEFAULT_TABLE table=test-table [id=7e99fd5c735e4638b4695fb5d0e99c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.417850 10016 log.cc:824] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.417920 9641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 055ee8d1e31f4f2c88df9462fa383d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.417896 10015 log.cc:824] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.418553 10015 tablet_bootstrap.cc:492] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: No bootstrap required, opened a new log
I20260923 22:08:13.418612 10016 tablet_bootstrap.cc:492] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: No bootstrap required, opened a new log
I20260923 22:08:13.418638 10015 ts_tablet_manager.cc:1397] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:13.418677 10016 ts_tablet_manager.cc:1397] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:13.420207 10015 raft_consensus.cc:348] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } }
I20260923 22:08:13.420358 10015 raft_consensus.cc:374] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.420400 10015 raft_consensus.cc:729] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f491a37f2184bcd83dba05de659aa75, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.420523 10015 consensus_queue.cc:260] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } }
I20260923 22:08:13.420823 9987 heartbeater.cc:503] Master 127.1.244.254:37371 was elected leader, sending a full tablet report...
I20260923 22:08:13.420789 10016 raft_consensus.cc:348] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } }
I20260923 22:08:13.420960 10016 raft_consensus.cc:374] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.421005 10016 raft_consensus.cc:729] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230612f73ece4f33aadd80548bfbaa84, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.420809 10015 ts_tablet_manager.cc:1428] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:08:13.421110 10016 consensus_queue.cc:260] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } }
I20260923 22:08:13.421360 10016 ts_tablet_manager.cc:1428] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:08:13.421402 9847 heartbeater.cc:503] Master 127.1.244.254:37371 was elected leader, sending a full tablet report...
I20260923 22:08:13.421906 10021 tablet_bootstrap.cc:492] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: Bootstrap starting.
I20260923 22:08:13.422477 10021 tablet_bootstrap.cc:654] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.422705 10021 log.cc:824] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.423298 10021 tablet_bootstrap.cc:492] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: No bootstrap required, opened a new log
I20260923 22:08:13.423348 10021 ts_tablet_manager.cc:1397] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:13.424791 10021 raft_consensus.cc:348] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } }
I20260923 22:08:13.424911 10021 raft_consensus.cc:374] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.424935 10021 raft_consensus.cc:729] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3989ca76c34afc80f4bd1a98106943, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.425047 10021 consensus_queue.cc:260] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } }
I20260923 22:08:13.425257 10021 ts_tablet_manager.cc:1428] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:13.425331 9707 heartbeater.cc:503] Master 127.1.244.254:37371 was elected leader, sending a full tablet report...
W20260923 22:08:13.457634 9848 tablet.cc:2369] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:08:13.544510 9708 tablet.cc:2369] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:13.588085 10020 raft_consensus.cc:482] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:13.588277 10020 raft_consensus.cc:504] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } }
I20260923 22:08:13.588713 10020 leader_election.cc:288] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851), 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567)
I20260923 22:08:13.591521 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.588961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:53405 (local address 127.1.244.195:36567)
0923 22:08:13.589008 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.589010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.589146 (+ 136us) server_negotiation.cc:407] Connection header received
0923 22:08:13.589208 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.589209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.589218 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.589258 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.589426 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.589888 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.590795 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.591157 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.591253 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.591260 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.591414 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.591457 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:08:13.591528 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.588891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:36567 (local address 127.1.244.194:53405)
0923 22:08:13.589057 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.589067 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.589147 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.589269 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.589270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.589272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.589354 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.589356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.589976 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.589977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.590698 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.590700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.591255 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.591256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.591336 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.591339 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.591408 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.591447 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
I20260923 22:08:13.591897 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.588904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:59101 (local address 127.1.244.193:41851)
0923 22:08:13.588977 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.588979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.588989 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:13.589066 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.589067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.589085 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.589162 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.589310 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.589792 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.590727 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.591092 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.591176 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.591185 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.591758 (+ 573us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.591802 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:08:13.591892 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.588879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41851 (local address 127.1.244.194:59101)
0923 22:08:13.588934 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.588945 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.588989 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.589168 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.589169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.589170 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.589239 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.589241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.589879 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.589880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.590619 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.590621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.591183 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.591184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.591668 (+ 484us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.591672 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.591749 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.591797 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":2}
I20260923 22:08:13.592064 9941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "055ee8d1e31f4f2c88df9462fa383d25" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f491a37f2184bcd83dba05de659aa75" is_pre_election: true
I20260923 22:08:13.592237 9941 raft_consensus.cc:2454] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 0.
I20260923 22:08:13.592285 9661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "055ee8d1e31f4f2c88df9462fa383d25" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3989ca76c34afc80f4bd1a98106943" is_pre_election: true
I20260923 22:08:13.592429 9661 raft_consensus.cc:2454] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 0.
I20260923 22:08:13.592468 9732 leader_election.cc:302] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230612f73ece4f33aadd80548bfbaa84, 9f491a37f2184bcd83dba05de659aa75; no voters:
I20260923 22:08:13.592578 10020 raft_consensus.cc:2781] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:13.592646 10020 raft_consensus.cc:482] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:13.592670 10020 raft_consensus.cc:3037] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.593345 10020 raft_consensus.cc:504] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } }
I20260923 22:08:13.593488 10020 leader_election.cc:288] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 election: Requested vote from peers 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851), 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567)
I20260923 22:08:13.593632 9661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "055ee8d1e31f4f2c88df9462fa383d25" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3989ca76c34afc80f4bd1a98106943"
I20260923 22:08:13.593653 9941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "055ee8d1e31f4f2c88df9462fa383d25" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f491a37f2184bcd83dba05de659aa75"
I20260923 22:08:13.593712 9661 raft_consensus.cc:3037] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.593736 9941 raft_consensus.cc:3037] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.594430 9941 raft_consensus.cc:2454] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 1.
I20260923 22:08:13.594432 9661 raft_consensus.cc:2454] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 1.
I20260923 22:08:13.594595 9734 leader_election.cc:302] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230612f73ece4f33aadd80548bfbaa84, 7d3989ca76c34afc80f4bd1a98106943; no voters:
I20260923 22:08:13.594671 10020 raft_consensus.cc:2781] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:13.594796 10020 raft_consensus.cc:686] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 1 LEADER]: Becoming Leader. State: Replica: 230612f73ece4f33aadd80548bfbaa84, State: Running, Role: LEADER
I20260923 22:08:13.594902 10020 consensus_queue.cc:237] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } }
I20260923 22:08:13.595764 9503 catalog_manager.cc:5495] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } health_report { overall_health: HEALTHY } } }
W20260923 22:08:13.616731 9988 tablet.cc:2369] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:13.637488 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.635135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39948 (local address 127.1.244.194:38869)
0923 22:08:13.635202 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.635204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.635230 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:08:13.635264 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.635265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.635282 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.635337 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:13.635514 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.636055 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.636788 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.636975 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.637026 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.637155 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:13.637268 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:13.637328 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.637358 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.637374 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:08:13.639418 9941 raft_consensus.cc:1261] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Refusing update from remote peer 230612f73ece4f33aadd80548bfbaa84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:13.639463 9661 raft_consensus.cc:1261] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Refusing update from remote peer 230612f73ece4f33aadd80548bfbaa84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:13.639776 10020 consensus_queue.cc:1035] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:13.639902 10025 consensus_queue.cc:1035] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:13.642485 10030 mvcc.cc:205] Tried to move back new op lower bound from 7332664498743992320 to 7332664498565447680. Current Snapshot: MvccSnapshot[applied={T|T < 7332664498743992320}]
I20260923 22:08:13.643292 10029 mvcc.cc:205] Tried to move back new op lower bound from 7332664498743992320 to 7332664498565447680. Current Snapshot: MvccSnapshot[applied={T|T < 7332664498743992320 or (T in {7332664498743992320})}]
I20260923 22:08:13.644908 9504 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:13.645018 9504 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:13.645768 9504 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02: Sending DeleteTablet for 3 replicas of tablet 055ee8d1e31f4f2c88df9462fa383d25
I20260923 22:08:13.646117 9641 tablet_service.cc:1514] Processing DeleteTablet for tablet 055ee8d1e31f4f2c88df9462fa383d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:13 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:45700
I20260923 22:08:13.646102 9921 tablet_service.cc:1514] Processing DeleteTablet for tablet 055ee8d1e31f4f2c88df9462fa383d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:13 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44578
I20260923 22:08:13.646116 9781 tablet_service.cc:1514] Processing DeleteTablet for tablet 055ee8d1e31f4f2c88df9462fa383d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:13 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:39938
I20260923 22:08:13.646363 10037 tablet_replica.cc:330] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: stopping tablet replica
I20260923 22:08:13.646481 10038 tablet_replica.cc:330] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: stopping tablet replica
I20260923 22:08:13.646481 10037 raft_consensus.cc:2229] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:13.646584 10037 raft_consensus.cc:2258] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:13.646526 9504 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:13.646661 9504 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:13.646785 10038 raft_consensus.cc:2229] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:13.646922 10038 raft_consensus.cc:2258] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:13.647053 10037 ts_tablet_manager.cc:1905] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:13.647430 10038 ts_tablet_manager.cc:1905] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:13.648284 9781 tablet_service.cc:1467] Processing CreateTablet for tablet 179ebfcb85a3449db6ed90c9630134f0 (DEFAULT_TABLE table=test-table [id=6e02fac54e1c4d058ea504ff738e863f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.648710 9921 tablet_service.cc:1467] Processing CreateTablet for tablet 179ebfcb85a3449db6ed90c9630134f0 (DEFAULT_TABLE table=test-table [id=6e02fac54e1c4d058ea504ff738e863f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.648833 9921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 179ebfcb85a3449db6ed90c9630134f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.648873 10037 ts_tablet_manager.cc:1918] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:13.648940 10037 log.cc:1196] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/055ee8d1e31f4f2c88df9462fa383d25
I20260923 22:08:13.649184 10037 ts_tablet_manager.cc:1939] T 055ee8d1e31f4f2c88df9462fa383d25 P 7d3989ca76c34afc80f4bd1a98106943: Deleting consensus metadata
I20260923 22:08:13.650096 10015 tablet_bootstrap.cc:492] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: Bootstrap starting.
I20260923 22:08:13.650161 9781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 179ebfcb85a3449db6ed90c9630134f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.650530 10015 tablet_bootstrap.cc:654] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.651185 10015 tablet_bootstrap.cc:492] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: No bootstrap required, opened a new log
I20260923 22:08:13.651190 10016 tablet_bootstrap.cc:492] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: Bootstrap starting.
I20260923 22:08:13.651232 10015 ts_tablet_manager.cc:1397] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:13.651360 9490 catalog_manager.cc:4841] TS 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851): tablet 055ee8d1e31f4f2c88df9462fa383d25 (table test-table [id=7e99fd5c735e4638b4695fb5d0e99c57]) successfully deleted
I20260923 22:08:13.651381 10015 raft_consensus.cc:348] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.651443 10015 raft_consensus.cc:374] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.651479 10015 raft_consensus.cc:729] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f491a37f2184bcd83dba05de659aa75, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.651522 10015 consensus_queue.cc:260] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.651535 9641 tablet_service.cc:1467] Processing CreateTablet for tablet 179ebfcb85a3449db6ed90c9630134f0 (DEFAULT_TABLE table=test-table [id=6e02fac54e1c4d058ea504ff738e863f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.651633 10016 tablet_bootstrap.cc:654] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.651633 10015 ts_tablet_manager.cc:1428] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.651643 9641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 179ebfcb85a3449db6ed90c9630134f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.652433 10016 tablet_bootstrap.cc:492] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: No bootstrap required, opened a new log
I20260923 22:08:13.652486 10016 ts_tablet_manager.cc:1397] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:13.652606 10016 raft_consensus.cc:348] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.652684 10016 raft_consensus.cc:374] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.652706 10016 raft_consensus.cc:729] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230612f73ece4f33aadd80548bfbaa84, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.652756 10016 consensus_queue.cc:260] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.652855 10016 ts_tablet_manager.cc:1428] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:13.652875 10021 tablet_bootstrap.cc:492] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: Bootstrap starting.
I20260923 22:08:13.653259 10021 tablet_bootstrap.cc:654] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.653889 10021 tablet_bootstrap.cc:492] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: No bootstrap required, opened a new log
I20260923 22:08:13.653945 10021 ts_tablet_manager.cc:1397] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:13.654302 10021 raft_consensus.cc:348] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.654383 10021 raft_consensus.cc:374] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.654405 10021 raft_consensus.cc:729] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3989ca76c34afc80f4bd1a98106943, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.654455 10038 ts_tablet_manager.cc:1918] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:13.654457 10021 consensus_queue.cc:260] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.654531 10038 log.cc:1196] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/055ee8d1e31f4f2c88df9462fa383d25
I20260923 22:08:13.654560 10021 ts_tablet_manager.cc:1428] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:13.654834 10038 ts_tablet_manager.cc:1939] T 055ee8d1e31f4f2c88df9462fa383d25 P 230612f73ece4f33aadd80548bfbaa84: Deleting consensus metadata
I20260923 22:08:13.655210 10039 tablet_replica.cc:330] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: stopping tablet replica
I20260923 22:08:13.655270 9491 catalog_manager.cc:4841] TS 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194:38869): tablet 055ee8d1e31f4f2c88df9462fa383d25 (table test-table [id=7e99fd5c735e4638b4695fb5d0e99c57]) successfully deleted
I20260923 22:08:13.655321 10039 raft_consensus.cc:2229] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:13.655431 10039 raft_consensus.cc:2258] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:13.655840 10039 ts_tablet_manager.cc:1905] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:13.657200 10039 ts_tablet_manager.cc:1918] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:13.657289 10039 log.cc:1196] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/055ee8d1e31f4f2c88df9462fa383d25
I20260923 22:08:13.657562 10039 ts_tablet_manager.cc:1939] T 055ee8d1e31f4f2c88df9462fa383d25 P 9f491a37f2184bcd83dba05de659aa75: Deleting consensus metadata
I20260923 22:08:13.657938 9488 catalog_manager.cc:4841] TS 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567): tablet 055ee8d1e31f4f2c88df9462fa383d25 (table test-table [id=7e99fd5c735e4638b4695fb5d0e99c57]) successfully deleted
I20260923 22:08:13.875685 10019 raft_consensus.cc:482] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:13.875789 10019 raft_consensus.cc:504] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.876216 10019 leader_election.cc:288] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851), 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194:38869)
I20260923 22:08:13.879479 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.876523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:39801 (local address 127.1.244.194:38869)
0923 22:08:13.876574 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.876577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.876654 (+ 77us) server_negotiation.cc:407] Connection header received
0923 22:08:13.876718 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.876720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.876745 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.876824 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.877013 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.877509 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.878671 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.879122 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.879211 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.879220 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.879370 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.879403 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:13.879492 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.876422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:38869 (local address 127.1.244.195:39801)
0923 22:08:13.876558 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.876572 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.876652 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.876840 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.876841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.876843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.876905 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.876907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.877588 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.877589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.878564 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.878567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.879211 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.879212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.879287 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.879290 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.879362 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.879403 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":78,"thread_start_us":69,"threads_started":1}
I20260923 22:08:13.879536 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.876615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:36835 (local address 127.1.244.193:41851)
0923 22:08:13.876657 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.876660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.876670 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:13.876740 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.876741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.876757 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.876825 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.877037 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.877568 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.878682 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.879152 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.879260 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.879269 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.879422 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.879462 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:08:13.879603 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.876443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41851 (local address 127.1.244.195:36835)
0923 22:08:13.876583 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.876593 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.876675 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.876831 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.876832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.876833 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.876932 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.876935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.877660 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.877661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.878570 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.878572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.879261 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.879262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.879340 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.879343 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.879415 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.879459 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260923 22:08:13.879739 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179ebfcb85a3449db6ed90c9630134f0" candidate_uuid: "9f491a37f2184bcd83dba05de659aa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "230612f73ece4f33aadd80548bfbaa84" is_pre_election: true
I20260923 22:08:13.879851 9801 raft_consensus.cc:2454] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f491a37f2184bcd83dba05de659aa75 in term 0.
I20260923 22:08:13.880050 9875 leader_election.cc:302] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230612f73ece4f33aadd80548bfbaa84, 9f491a37f2184bcd83dba05de659aa75; no voters:
I20260923 22:08:13.880075 9661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179ebfcb85a3449db6ed90c9630134f0" candidate_uuid: "9f491a37f2184bcd83dba05de659aa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3989ca76c34afc80f4bd1a98106943" is_pre_election: true
I20260923 22:08:13.880156 9661 raft_consensus.cc:2454] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f491a37f2184bcd83dba05de659aa75 in term 0.
I20260923 22:08:13.880239 10019 raft_consensus.cc:2781] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:13.880277 10019 raft_consensus.cc:482] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:13.880296 10019 raft_consensus.cc:3037] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.880945 10019 raft_consensus.cc:504] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.881100 10019 leader_election.cc:288] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [CANDIDATE]: Term 1 election: Requested vote from peers 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851), 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194:38869)
I20260923 22:08:13.881229 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179ebfcb85a3449db6ed90c9630134f0" candidate_uuid: "9f491a37f2184bcd83dba05de659aa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "230612f73ece4f33aadd80548bfbaa84"
I20260923 22:08:13.881304 9801 raft_consensus.cc:3037] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.881315 9661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179ebfcb85a3449db6ed90c9630134f0" candidate_uuid: "9f491a37f2184bcd83dba05de659aa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3989ca76c34afc80f4bd1a98106943"
I20260923 22:08:13.881409 9661 raft_consensus.cc:3037] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.882126 9801 raft_consensus.cc:2454] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f491a37f2184bcd83dba05de659aa75 in term 1.
I20260923 22:08:13.882123 9661 raft_consensus.cc:2454] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f491a37f2184bcd83dba05de659aa75 in term 1.
I20260923 22:08:13.882305 9874 leader_election.cc:302] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d3989ca76c34afc80f4bd1a98106943, 9f491a37f2184bcd83dba05de659aa75; no voters:
I20260923 22:08:13.882432 10019 raft_consensus.cc:2781] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:13.882581 10019 raft_consensus.cc:686] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 1 LEADER]: Becoming Leader. State: Replica: 9f491a37f2184bcd83dba05de659aa75, State: Running, Role: LEADER
I20260923 22:08:13.882663 10019 consensus_queue.cc:237] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.883294 9504 catalog_manager.cc:5495] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "9f491a37f2184bcd83dba05de659aa75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } health_report { overall_health: HEALTHY } } }
Sep 23 22:08:13 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:13.979035 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.972337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.1.244.254:37371)
0923 22:08:13.972514 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.972516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.972572 (+ 56us) server_negotiation.cc:407] Connection header received
0923 22:08:13.972634 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.972635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.972645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.972729 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.973180 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.973681 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.974343 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.974535 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.974585 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.976635 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.976637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.976637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:13.976643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.977980 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.978351 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.978352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.978352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.978406 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:13.978594 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:13.978595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:13.978596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:13.978726 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.978790 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.978892 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.978963 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260923 22:08:13.979193 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.972293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37371 (local address 127.0.0.1:55464)
0923 22:08:13.972483 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.972502 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.972575 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.972741 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.972742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.973058 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.973110 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.973112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.973751 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.973751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.974270 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.974271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.974603 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.974603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.974644 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.975130 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:13.975134 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.976559 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.978076 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.978083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.978085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.978292 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.978465 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:13.978466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:13.978466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:13.978523 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:13.978782 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.978787 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.978826 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.978875 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.979030 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":65,"thread_start_us":93,"threads_started":1}
I20260923 22:08:13.981387 9504 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:55464:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:08:13.981575 9504 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:13.982933 9781 tablet_service.cc:1467] Processing CreateTablet for tablet 05a45ac6ccd54fefb71fdf1e7086fe7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5f99fe05f06c4c74b3985f4f3e051241]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:13.982949 9641 tablet_service.cc:1467] Processing CreateTablet for tablet 05a45ac6ccd54fefb71fdf1e7086fe7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5f99fe05f06c4c74b3985f4f3e051241]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:13.983100 9641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a45ac6ccd54fefb71fdf1e7086fe7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.983100 9781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a45ac6ccd54fefb71fdf1e7086fe7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.984385 10021 tablet_bootstrap.cc:492] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: Bootstrap starting.
I20260923 22:08:13.984504 9921 tablet_service.cc:1467] Processing CreateTablet for tablet 05a45ac6ccd54fefb71fdf1e7086fe7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5f99fe05f06c4c74b3985f4f3e051241]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:13.984639 9921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a45ac6ccd54fefb71fdf1e7086fe7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.984813 10021 tablet_bootstrap.cc:654] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.985446 10021 tablet_bootstrap.cc:492] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: No bootstrap required, opened a new log
I20260923 22:08:13.985507 10021 ts_tablet_manager.cc:1397] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:13.985653 10021 raft_consensus.cc:348] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.985726 10021 raft_consensus.cc:374] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.985746 10021 raft_consensus.cc:729] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3989ca76c34afc80f4bd1a98106943, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.985790 10021 consensus_queue.cc:260] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.985894 10021 ts_tablet_manager.cc:1428] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:13.985957 10023 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:13.986089 10023 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:13.986263 10016 tablet_bootstrap.cc:492] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: Bootstrap starting.
I20260923 22:08:13.986608 10016 tablet_bootstrap.cc:654] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.986814 10015 tablet_bootstrap.cc:492] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: Bootstrap starting.
I20260923 22:08:13.987157 10016 tablet_bootstrap.cc:492] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: No bootstrap required, opened a new log
I20260923 22:08:13.987202 10015 tablet_bootstrap.cc:654] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.987215 10016 ts_tablet_manager.cc:1397] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:13.987326 10016 raft_consensus.cc:348] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.987421 10016 raft_consensus.cc:374] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.987447 10016 raft_consensus.cc:729] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230612f73ece4f33aadd80548bfbaa84, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.987502 10016 consensus_queue.cc:260] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.987643 10016 ts_tablet_manager.cc:1428] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.987655 10020 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:13.987742 10020 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:13.988054 10015 tablet_bootstrap.cc:492] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: No bootstrap required, opened a new log
I20260923 22:08:13.988106 10015 ts_tablet_manager.cc:1397] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:13.988238 10015 raft_consensus.cc:348] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.988288 10015 raft_consensus.cc:374] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.988301 10015 raft_consensus.cc:729] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f491a37f2184bcd83dba05de659aa75, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.988356 10015 consensus_queue.cc:260] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:13.988451 10015 ts_tablet_manager.cc:1428] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.988489 10019 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:13.988556 10019 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:14.143941 10020 raft_consensus.cc:482] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:14.144040 10020 raft_consensus.cc:504] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.144215 10020 leader_election.cc:288] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851), 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567)
I20260923 22:08:14.144456 9661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a45ac6ccd54fefb71fdf1e7086fe7a" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3989ca76c34afc80f4bd1a98106943" is_pre_election: true
I20260923 22:08:14.144500 9941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a45ac6ccd54fefb71fdf1e7086fe7a" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f491a37f2184bcd83dba05de659aa75" is_pre_election: true
I20260923 22:08:14.144567 9661 raft_consensus.cc:2454] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 0.
I20260923 22:08:14.144618 9941 raft_consensus.cc:2454] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 0.
I20260923 22:08:14.144758 9732 leader_election.cc:302] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230612f73ece4f33aadd80548bfbaa84, 9f491a37f2184bcd83dba05de659aa75; no voters:
I20260923 22:08:14.144860 10020 raft_consensus.cc:2781] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:14.144901 10020 raft_consensus.cc:482] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:14.144919 10020 raft_consensus.cc:3037] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.145560 10020 raft_consensus.cc:504] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.145690 10020 leader_election.cc:288] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 election: Requested vote from peers 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851), 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567)
I20260923 22:08:14.145831 9661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a45ac6ccd54fefb71fdf1e7086fe7a" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3989ca76c34afc80f4bd1a98106943"
I20260923 22:08:14.145854 9941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a45ac6ccd54fefb71fdf1e7086fe7a" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f491a37f2184bcd83dba05de659aa75"
I20260923 22:08:14.145918 9661 raft_consensus.cc:3037] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.145941 9941 raft_consensus.cc:3037] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.146528 9941 raft_consensus.cc:2454] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 1.
I20260923 22:08:14.146561 9661 raft_consensus.cc:2454] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 1.
I20260923 22:08:14.146713 9734 leader_election.cc:302] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230612f73ece4f33aadd80548bfbaa84, 7d3989ca76c34afc80f4bd1a98106943; no voters:
I20260923 22:08:14.146816 10020 raft_consensus.cc:2781] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:14.146938 10020 raft_consensus.cc:686] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [term 1 LEADER]: Becoming Leader. State: Replica: 230612f73ece4f33aadd80548bfbaa84, State: Running, Role: LEADER
I20260923 22:08:14.147011 10020 consensus_queue.cc:237] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.147317 10057 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: TxnStatusTablet state changed. Reason: New leader 230612f73ece4f33aadd80548bfbaa84. Latest consensus state: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:14.147444 10057 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:14.147642 10059 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:08:14.147655 9504 catalog_manager.cc:5495] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:14.147778 10059 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:08:14.171293 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.168568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:38869 (local address 127.0.0.1:39954)
0923 22:08:14.168679 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.168691 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.168758 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.168895 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.168896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.168897 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:14.168973 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.168976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.169868 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.169869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.170437 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.170439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.170904 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.170905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.170962 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.170977 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.171154 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.171160 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:08:14.171195 (+ 35us) client_negotiation.cc:240] Negotiation successful
0923 22:08:14.171235 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260923 22:08:14.171295 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.168636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39954 (local address 127.1.244.194:38869)
0923 22:08:14.168704 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.168706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.168741 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:08:14.168812 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.168813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.168824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.168882 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:14.169060 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.169787 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.170518 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.170822 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.170893 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.171054 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.171090 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.171143 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.171198 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.171218 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:08:14.172366 9941 raft_consensus.cc:1261] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Refusing update from remote peer 230612f73ece4f33aadd80548bfbaa84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:14.172367 9661 raft_consensus.cc:1261] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Refusing update from remote peer 230612f73ece4f33aadd80548bfbaa84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:14.172598 10057 consensus_queue.cc:1035] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:14.172726 10020 consensus_queue.cc:1035] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:14.173229 10023 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: TxnStatusTablet state changed. Reason: New leader 230612f73ece4f33aadd80548bfbaa84. Latest consensus state: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:14.173264 10019 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: TxnStatusTablet state changed. Reason: New leader 230612f73ece4f33aadd80548bfbaa84. Latest consensus state: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:14.173321 10023 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:14.173341 10019 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:14.174185 10020 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:14.174217 10057 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:14.174268 10020 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:14.174285 10057 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 230612f73ece4f33aadd80548bfbaa84: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:14.174924 10019 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:14.174932 10023 tablet_replica.cc:439] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } } }
I20260923 22:08:14.175005 10023 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 7d3989ca76c34afc80f4bd1a98106943: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:14.175000 10019 tablet_replica.cc:442] T 05a45ac6ccd54fefb71fdf1e7086fe7a P 9f491a37f2184bcd83dba05de659aa75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:14.175307 10062 mvcc.cc:205] Tried to move back new op lower bound from 7332664500928155648 to 7332664500826439680. Current Snapshot: MvccSnapshot[applied={T|T < 7332664500928155648 or (T in {7332664500928155648})}]
I20260923 22:08:14.178881 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.176184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44580 (local address 127.1.244.195:36567)
0923 22:08:14.176266 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.176268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.176276 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:14.176368 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.176368 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.176384 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.176440 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:14.176570 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.177010 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.178139 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.178345 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.178424 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.178592 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.178684 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.178733 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.178794 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.178810 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"thread_start_us":46,"threads_started":1}
I20260923 22:08:14.183661 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.181263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:35137 (local address 127.1.244.194:38869)
0923 22:08:14.181329 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.181331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.181469 (+ 138us) server_negotiation.cc:407] Connection header received
0923 22:08:14.181496 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.181496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.181506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.181566 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:14.181694 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.182164 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.182941 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.183135 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.183212 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.183390 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.183429 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.183492 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.183577 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.183598 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:08:14.183704 10076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.181199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:38869 (local address 127.1.244.195:35137)
0923 22:08:14.181403 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.181411 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.181461 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.181577 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.181578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.181579 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:14.181624 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.181625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.182249 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.182250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.182838 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.182840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.183212 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.183213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.183284 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.183296 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.183494 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.183496 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:14.183575 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:08:14.183619 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260923 22:08:14.185659 9661 raft_consensus.cc:1261] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Refusing update from remote peer 9f491a37f2184bcd83dba05de659aa75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:14.185799 9801 raft_consensus.cc:1261] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Refusing update from remote peer 9f491a37f2184bcd83dba05de659aa75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:14.185894 10045 consensus_queue.cc:1035] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:14.186034 10019 consensus_queue.cc:1035] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:14.193928 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.191619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:51985 (local address 127.1.244.195:36567)
0923 22:08:14.191656 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.191658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.191665 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:14.191687 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.191688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.191697 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.191749 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:14.191862 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.192294 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.193358 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.193536 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.193585 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.193756 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.193780 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.193816 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.193868 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.193883 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:14.194170 10080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.191412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:36567 (local address 127.1.244.194:51985)
0923 22:08:14.191577 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.191588 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.191638 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.191753 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.191753 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.191756 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:14.191797 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.191799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.192362 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.192362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.193269 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.193272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.193605 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.193606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.193663 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.193678 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.193817 (+ 139us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.193818 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:14.193867 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:08:14.193910 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":82,"threads_started":1}
I20260923 22:08:14.200834 9502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:14.200927 9502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:14.201556 9502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02: Sending DeleteTablet for 3 replicas of tablet 179ebfcb85a3449db6ed90c9630134f0
I20260923 22:08:14.201792 9781 tablet_service.cc:1514] Processing DeleteTablet for tablet 179ebfcb85a3449db6ed90c9630134f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:39938
I20260923 22:08:14.201929 9921 tablet_service.cc:1514] Processing DeleteTablet for tablet 179ebfcb85a3449db6ed90c9630134f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44578
I20260923 22:08:14.201973 9641 tablet_service.cc:1514] Processing DeleteTablet for tablet 179ebfcb85a3449db6ed90c9630134f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:45700
I20260923 22:08:14.202093 10084 tablet_replica.cc:330] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: stopping tablet replica
I20260923 22:08:14.202180 10084 raft_consensus.cc:2229] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:14.202255 10084 raft_consensus.cc:2258] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:14.202548 10084 ts_tablet_manager.cc:1905] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:14.204299 10084 ts_tablet_manager.cc:1918] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:14.204423 10084 log.cc:1196] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/179ebfcb85a3449db6ed90c9630134f0
I20260923 22:08:14.204720 10084 ts_tablet_manager.cc:1939] T 179ebfcb85a3449db6ed90c9630134f0 P 230612f73ece4f33aadd80548bfbaa84: Deleting consensus metadata
I20260923 22:08:14.205072 9491 catalog_manager.cc:4841] TS 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194:38869): tablet 179ebfcb85a3449db6ed90c9630134f0 (table test-table [id=6e02fac54e1c4d058ea504ff738e863f]) successfully deleted
I20260923 22:08:14.205279 10086 tablet_replica.cc:330] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: stopping tablet replica
I20260923 22:08:14.205380 10086 raft_consensus.cc:2229] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:14.205564 10086 raft_consensus.cc:2258] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:14.205826 10086 ts_tablet_manager.cc:1905] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:14.206276 10085 tablet_replica.cc:330] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: stopping tablet replica
I20260923 22:08:14.206342 10085 raft_consensus.cc:2229] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:14.206434 10085 raft_consensus.cc:2258] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:14.206679 10085 ts_tablet_manager.cc:1905] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:14.207708 10086 ts_tablet_manager.cc:1918] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:14.207813 10086 log.cc:1196] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/179ebfcb85a3449db6ed90c9630134f0
I20260923 22:08:14.208103 10086 ts_tablet_manager.cc:1939] T 179ebfcb85a3449db6ed90c9630134f0 P 7d3989ca76c34afc80f4bd1a98106943: Deleting consensus metadata
I20260923 22:08:14.208362 10085 ts_tablet_manager.cc:1918] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:14.208441 10085 log.cc:1196] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/179ebfcb85a3449db6ed90c9630134f0
I20260923 22:08:14.208434 9490 catalog_manager.cc:4841] TS 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851): tablet 179ebfcb85a3449db6ed90c9630134f0 (table test-table [id=6e02fac54e1c4d058ea504ff738e863f]) successfully deleted
I20260923 22:08:14.208666 10085 ts_tablet_manager.cc:1939] T 179ebfcb85a3449db6ed90c9630134f0 P 9f491a37f2184bcd83dba05de659aa75: Deleting consensus metadata
I20260923 22:08:14.208969 9488 catalog_manager.cc:4841] TS 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567): tablet 179ebfcb85a3449db6ed90c9630134f0 (table test-table [id=6e02fac54e1c4d058ea504ff738e863f]) successfully deleted
Sep 23 22:08:14 dist-test-slave-b5jz krb5kdc[9459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:08:14 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:14 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:08:14.239944 10053 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:14.232795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.1.244.254:37371)
0923 22:08:14.232845 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.232847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.232997 (+ 150us) server_negotiation.cc:407] Connection header received
0923 22:08:14.233052 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.233053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.233079 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.233149 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:14.233880 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.234692 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.235851 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.236175 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.236260 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.239878 (+ 3618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55466: BlockingRecv error: recv got EOF from 127.0.0.1:55466 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 23 22:08:14 dist-test-slave-b5jz krb5kdc[9459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260923 22:08:14.250382 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.241034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55476 (local address 127.1.244.254:37371)
0923 22:08:14.241069 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.241070 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.241314 (+ 244us) server_negotiation.cc:407] Connection header received
0923 22:08:14.241362 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.241363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.241373 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.241434 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:14.242008 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.242747 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.243816 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.244114 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.244180 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.247366 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:14.247367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:14.247368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:14.247375 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:14.249162 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:14.249557 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:14.249558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:14.249558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:14.249610 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:14.249794 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:14.249795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:14.249795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:14.249920 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:14.250005 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.250205 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.250285 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:08:14.252581 9502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:14.252758 9502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:14.254074 9641 tablet_service.cc:1467] Processing CreateTablet for tablet 06c0b776d02445cca45f4620f3ad3056 (DEFAULT_TABLE table=test-table [id=e14da5e18f694fa0bb2f9c1e1ba787c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:14.254206 9641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c0b776d02445cca45f4620f3ad3056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.254217 9781 tablet_service.cc:1467] Processing CreateTablet for tablet 06c0b776d02445cca45f4620f3ad3056 (DEFAULT_TABLE table=test-table [id=e14da5e18f694fa0bb2f9c1e1ba787c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:14.254340 9781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c0b776d02445cca45f4620f3ad3056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.255529 10021 tablet_bootstrap.cc:492] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: Bootstrap starting.
I20260923 22:08:14.255764 9921 tablet_service.cc:1467] Processing CreateTablet for tablet 06c0b776d02445cca45f4620f3ad3056 (DEFAULT_TABLE table=test-table [id=e14da5e18f694fa0bb2f9c1e1ba787c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:14.255874 9921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c0b776d02445cca45f4620f3ad3056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.255982 10021 tablet_bootstrap.cc:654] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.257205 10015 tablet_bootstrap.cc:492] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: Bootstrap starting.
I20260923 22:08:14.257241 10016 tablet_bootstrap.cc:492] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: Bootstrap starting.
I20260923 22:08:14.257563 10016 tablet_bootstrap.cc:654] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.257653 10015 tablet_bootstrap.cc:654] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.258322 10015 tablet_bootstrap.cc:492] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: No bootstrap required, opened a new log
I20260923 22:08:14.258368 10015 ts_tablet_manager.cc:1397] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:14.258425 10016 tablet_bootstrap.cc:492] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: No bootstrap required, opened a new log
I20260923 22:08:14.258471 10016 ts_tablet_manager.cc:1397] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:08:14.258499 10015 raft_consensus.cc:348] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.258562 10015 raft_consensus.cc:374] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.258582 10015 raft_consensus.cc:729] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f491a37f2184bcd83dba05de659aa75, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.258584 10016 raft_consensus.cc:348] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.258625 10015 consensus_queue.cc:260] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.258647 10016 raft_consensus.cc:374] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.258668 10016 raft_consensus.cc:729] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230612f73ece4f33aadd80548bfbaa84, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.258713 10016 consensus_queue.cc:260] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.258728 10015 ts_tablet_manager.cc:1428] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.258797 10016 ts_tablet_manager.cc:1428] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.263999 10021 tablet_bootstrap.cc:492] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: No bootstrap required, opened a new log
I20260923 22:08:14.264062 10021 ts_tablet_manager.cc:1397] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260923 22:08:14.264213 10021 raft_consensus.cc:348] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.264282 10021 raft_consensus.cc:374] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.264302 10021 raft_consensus.cc:729] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3989ca76c34afc80f4bd1a98106943, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.264351 10021 consensus_queue.cc:260] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.264439 10021 ts_tablet_manager.cc:1428] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.279157 10067 raft_consensus.cc:482] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:14.279268 10067 raft_consensus.cc:504] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.279422 10067 leader_election.cc:288] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851), 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567)
I20260923 22:08:14.279649 9661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0b776d02445cca45f4620f3ad3056" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3989ca76c34afc80f4bd1a98106943" is_pre_election: true
I20260923 22:08:14.279743 9661 raft_consensus.cc:2454] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 0.
I20260923 22:08:14.279678 9941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0b776d02445cca45f4620f3ad3056" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f491a37f2184bcd83dba05de659aa75" is_pre_election: true
I20260923 22:08:14.279798 9941 raft_consensus.cc:2454] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 0.
I20260923 22:08:14.279896 9734 leader_election.cc:302] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230612f73ece4f33aadd80548bfbaa84, 7d3989ca76c34afc80f4bd1a98106943; no voters:
I20260923 22:08:14.279999 10067 raft_consensus.cc:2781] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:14.280031 10067 raft_consensus.cc:482] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:14.280051 10067 raft_consensus.cc:3037] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.280938 10067 raft_consensus.cc:504] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.281073 10067 leader_election.cc:288] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 election: Requested vote from peers 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851), 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567)
I20260923 22:08:14.281261 9941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0b776d02445cca45f4620f3ad3056" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f491a37f2184bcd83dba05de659aa75"
I20260923 22:08:14.281327 9941 raft_consensus.cc:3037] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.281359 9661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c0b776d02445cca45f4620f3ad3056" candidate_uuid: "230612f73ece4f33aadd80548bfbaa84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3989ca76c34afc80f4bd1a98106943"
I20260923 22:08:14.281445 9661 raft_consensus.cc:3037] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.282045 9941 raft_consensus.cc:2454] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 1.
I20260923 22:08:14.282193 9732 leader_election.cc:302] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230612f73ece4f33aadd80548bfbaa84, 9f491a37f2184bcd83dba05de659aa75; no voters:
I20260923 22:08:14.282230 9661 raft_consensus.cc:2454] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 230612f73ece4f33aadd80548bfbaa84 in term 1.
I20260923 22:08:14.282264 10067 raft_consensus.cc:2781] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:14.282295 10067 raft_consensus.cc:686] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 1 LEADER]: Becoming Leader. State: Replica: 230612f73ece4f33aadd80548bfbaa84, State: Running, Role: LEADER
I20260923 22:08:14.282334 10067 consensus_queue.cc:237] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.282912 9502 catalog_manager.cc:5495] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "230612f73ece4f33aadd80548bfbaa84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:14.291648 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.288540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.1.244.194:38869)
0923 22:08:14.288583 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.288586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.288669 (+ 83us) server_negotiation.cc:407] Connection header received
0923 22:08:14.288767 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.288768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.288779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.288808 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:14.289018 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.289716 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.290816 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.291110 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.291157 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.291365 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.291398 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.291426 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.291537 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.291580 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:08:14.292512 9941 raft_consensus.cc:1261] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Refusing update from remote peer 230612f73ece4f33aadd80548bfbaa84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:14.292539 9661 raft_consensus.cc:1261] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Refusing update from remote peer 230612f73ece4f33aadd80548bfbaa84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:14.292732 10057 consensus_queue.cc:1035] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:14.292827 10020 consensus_queue.cc:1035] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:14.295655 9503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:14.295739 9503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:14.296425 9503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02: Sending DeleteTablet for 3 replicas of tablet 06c0b776d02445cca45f4620f3ad3056
I20260923 22:08:14.296672 9641 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c0b776d02445cca45f4620f3ad3056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:45700
I20260923 22:08:14.296759 10086 tablet_replica.cc:330] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: stopping tablet replica
I20260923 22:08:14.296804 10086 raft_consensus.cc:2229] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:14.296856 10086 raft_consensus.cc:2258] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:14.296916 9921 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c0b776d02445cca45f4620f3ad3056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44578
I20260923 22:08:14.297027 9920 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c0b776d02445cca45f4620f3ad3056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44578
I20260923 22:08:14.297086 10085 tablet_replica.cc:330] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: stopping tablet replica
I20260923 22:08:14.297127 10085 raft_consensus.cc:2229] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:14.297137 10086 ts_tablet_manager.cc:1905] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:14.297163 10085 raft_consensus.cc:2258] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:14.297499 9781 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c0b776d02445cca45f4620f3ad3056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:39938
I20260923 22:08:14.297593 10084 tablet_replica.cc:330] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: stopping tablet replica
I20260923 22:08:14.297638 10084 raft_consensus.cc:2229] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:14.297683 10084 raft_consensus.cc:2258] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:14.297976 10084 ts_tablet_manager.cc:1905] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:14.298781 10086 ts_tablet_manager.cc:1918] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:14.298846 10086 log.cc:1196] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/06c0b776d02445cca45f4620f3ad3056
I20260923 22:08:14.299009 10086 ts_tablet_manager.cc:1939] T 06c0b776d02445cca45f4620f3ad3056 P 7d3989ca76c34afc80f4bd1a98106943: Deleting consensus metadata
I20260923 22:08:14.299345 10084 ts_tablet_manager.cc:1918] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:14.299350 9490 catalog_manager.cc:4841] TS 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851): tablet 06c0b776d02445cca45f4620f3ad3056 (table test-table [id=e14da5e18f694fa0bb2f9c1e1ba787c6]) successfully deleted
I20260923 22:08:14.299391 10084 log.cc:1196] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/06c0b776d02445cca45f4620f3ad3056
I20260923 22:08:14.299669 10084 ts_tablet_manager.cc:1939] T 06c0b776d02445cca45f4620f3ad3056 P 230612f73ece4f33aadd80548bfbaa84: Deleting consensus metadata
I20260923 22:08:14.300087 9491 catalog_manager.cc:4841] TS 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194:38869): tablet 06c0b776d02445cca45f4620f3ad3056 (table test-table [id=e14da5e18f694fa0bb2f9c1e1ba787c6]) successfully deleted
I20260923 22:08:14.300105 9503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:14.300287 9503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:14.300516 10085 ts_tablet_manager.cc:1905] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:14.301927 10085 ts_tablet_manager.cc:1918] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:14.301998 10085 log.cc:1196] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/06c0b776d02445cca45f4620f3ad3056
I20260923 22:08:14.302227 10085 ts_tablet_manager.cc:1939] T 06c0b776d02445cca45f4620f3ad3056 P 9f491a37f2184bcd83dba05de659aa75: Deleting consensus metadata
I20260923 22:08:14.302637 9488 catalog_manager.cc:4841] TS 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567): tablet 06c0b776d02445cca45f4620f3ad3056 (table test-table [id=e14da5e18f694fa0bb2f9c1e1ba787c6]) successfully deleted
W20260923 22:08:14.302778 9488 catalog_manager.cc:4808] TS 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567): delete failed for tablet 06c0b776d02445cca45f4620f3ad3056 because the tablet was not found. No further retry: Not found: Tablet not found: 06c0b776d02445cca45f4620f3ad3056
I20260923 22:08:14.302942 9781 tablet_service.cc:1467] Processing CreateTablet for tablet bf0a3045c73b4197a0ac13cff755fed8 (DEFAULT_TABLE table=test-table [id=aae682b5611e479581a3284b0536462f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:14.303012 9920 tablet_service.cc:1467] Processing CreateTablet for tablet bf0a3045c73b4197a0ac13cff755fed8 (DEFAULT_TABLE table=test-table [id=aae682b5611e479581a3284b0536462f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:14.303052 9781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf0a3045c73b4197a0ac13cff755fed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.303134 9920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf0a3045c73b4197a0ac13cff755fed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.304148 10016 tablet_bootstrap.cc:492] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: Bootstrap starting.
I20260923 22:08:14.304430 10016 tablet_bootstrap.cc:654] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.304543 10015 tablet_bootstrap.cc:492] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: Bootstrap starting.
I20260923 22:08:14.304729 9641 tablet_service.cc:1467] Processing CreateTablet for tablet bf0a3045c73b4197a0ac13cff755fed8 (DEFAULT_TABLE table=test-table [id=aae682b5611e479581a3284b0536462f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:14.304843 9641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf0a3045c73b4197a0ac13cff755fed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.304927 10015 tablet_bootstrap.cc:654] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.305562 10015 tablet_bootstrap.cc:492] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: No bootstrap required, opened a new log
I20260923 22:08:14.305615 10015 ts_tablet_manager.cc:1397] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:08:14.305750 10015 raft_consensus.cc:348] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.305824 10015 raft_consensus.cc:374] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.305845 10015 raft_consensus.cc:729] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f491a37f2184bcd83dba05de659aa75, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.305888 10015 consensus_queue.cc:260] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.305981 10015 ts_tablet_manager.cc:1428] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.306448 10016 tablet_bootstrap.cc:492] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: No bootstrap required, opened a new log
I20260923 22:08:14.306507 10016 ts_tablet_manager.cc:1397] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260923 22:08:14.306613 10016 raft_consensus.cc:348] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.306699 10016 raft_consensus.cc:374] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.306731 10016 raft_consensus.cc:729] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230612f73ece4f33aadd80548bfbaa84, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.306782 10016 consensus_queue.cc:260] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.306874 10016 ts_tablet_manager.cc:1428] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.307096 10021 tablet_bootstrap.cc:492] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943: Bootstrap starting.
I20260923 22:08:14.307737 10021 tablet_bootstrap.cc:654] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.308466 10021 tablet_bootstrap.cc:492] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943: No bootstrap required, opened a new log
I20260923 22:08:14.308521 10021 ts_tablet_manager.cc:1397] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:14.308657 10021 raft_consensus.cc:348] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.308732 10021 raft_consensus.cc:374] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.308754 10021 raft_consensus.cc:729] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3989ca76c34afc80f4bd1a98106943, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.308805 10021 consensus_queue.cc:260] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.308900 10021 ts_tablet_manager.cc:1428] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:14.422626 10023 raft_consensus.cc:482] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:14.422719 10023 raft_consensus.cc:504] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.423022 10023 leader_election.cc:288] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194:38869), 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567)
I20260923 22:08:14.425997 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.423313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:37253 (local address 127.1.244.195:36567)
0923 22:08:14.423384 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.423386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.423410 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:08:14.423483 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.423484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.423493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.423575 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:14.423749 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.424235 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.425252 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.425635 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.425716 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.425725 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.425881 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.425926 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:08:14.426010 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.423387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:36145 (local address 127.1.244.194:38869)
0923 22:08:14.423435 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.423437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.423467 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:08:14.423537 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.423538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.423569 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.423622 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:14.423786 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.424240 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.425262 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.425635 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.425726 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.425737 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.425911 (+ 174us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.425942 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:08:14.426069 10110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.423198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:36567 (local address 127.1.244.193:37253)
0923 22:08:14.423345 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.423358 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.423407 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.423585 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.423586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.423589 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:14.423668 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.423670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.424344 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.424345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.425163 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.425165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.425725 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.425726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.425807 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.425811 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:14.425876 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:08:14.425919 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":60,"threads_started":1}
I20260923 22:08:14.426241 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.423198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:38869 (local address 127.1.244.193:36145)
0923 22:08:14.423395 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.423407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.423477 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.423626 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.423627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.423629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:14.423711 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.423713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.424373 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.424374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.425167 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.425169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.425727 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.425728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.425824 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.425828 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:14.425893 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:08:14.425940 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":32,"thread_start_us":58,"threads_started":1}
I20260923 22:08:14.426321 9941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0a3045c73b4197a0ac13cff755fed8" candidate_uuid: "7d3989ca76c34afc80f4bd1a98106943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f491a37f2184bcd83dba05de659aa75" is_pre_election: true
I20260923 22:08:14.426410 9941 raft_consensus.cc:2454] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d3989ca76c34afc80f4bd1a98106943 in term 0.
I20260923 22:08:14.426496 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0a3045c73b4197a0ac13cff755fed8" candidate_uuid: "7d3989ca76c34afc80f4bd1a98106943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "230612f73ece4f33aadd80548bfbaa84" is_pre_election: true
I20260923 22:08:14.426582 9801 raft_consensus.cc:2454] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d3989ca76c34afc80f4bd1a98106943 in term 0.
I20260923 22:08:14.426597 9592 leader_election.cc:302] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d3989ca76c34afc80f4bd1a98106943, 9f491a37f2184bcd83dba05de659aa75; no voters:
I20260923 22:08:14.426753 10023 raft_consensus.cc:2781] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:14.426803 10023 raft_consensus.cc:482] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:14.426826 10023 raft_consensus.cc:3037] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.427485 10023 raft_consensus.cc:504] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.427646 10023 leader_election.cc:288] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [CANDIDATE]: Term 1 election: Requested vote from peers 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194:38869), 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567)
I20260923 22:08:14.427810 9941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0a3045c73b4197a0ac13cff755fed8" candidate_uuid: "7d3989ca76c34afc80f4bd1a98106943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f491a37f2184bcd83dba05de659aa75"
I20260923 22:08:14.427829 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0a3045c73b4197a0ac13cff755fed8" candidate_uuid: "7d3989ca76c34afc80f4bd1a98106943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "230612f73ece4f33aadd80548bfbaa84"
I20260923 22:08:14.427892 9801 raft_consensus.cc:3037] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.427898 9941 raft_consensus.cc:3037] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.428615 9801 raft_consensus.cc:2454] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d3989ca76c34afc80f4bd1a98106943 in term 1.
I20260923 22:08:14.428730 9941 raft_consensus.cc:2454] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d3989ca76c34afc80f4bd1a98106943 in term 1.
I20260923 22:08:14.428819 9595 leader_election.cc:302] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230612f73ece4f33aadd80548bfbaa84, 7d3989ca76c34afc80f4bd1a98106943; no voters:
I20260923 22:08:14.428942 10023 raft_consensus.cc:2781] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:14.429102 10023 raft_consensus.cc:686] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 LEADER]: Becoming Leader. State: Replica: 7d3989ca76c34afc80f4bd1a98106943, State: Running, Role: LEADER
I20260923 22:08:14.429174 10023 consensus_queue.cc:237] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } }
I20260923 22:08:14.429872 9503 catalog_manager.cc:5495] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "7d3989ca76c34afc80f4bd1a98106943" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3989ca76c34afc80f4bd1a98106943" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:14.453184 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.450787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.1.244.193:41851)
0923 22:08:14.450926 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.450927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.450933 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:08:14.450966 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.450966 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.450986 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.451049 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:14.451174 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.451752 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.452436 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.452651 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.452707 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.452818 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.452925 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.452978 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.453015 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.453031 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260923 22:08:14.458139 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.455687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53971 (local address 127.1.244.194:38869)
0923 22:08:14.455743 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.455745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.455754 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:14.455797 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.455798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.455810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.455860 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:14.456018 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.456481 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.457256 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.457490 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.457526 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.457876 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.457910 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.457983 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.458045 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.458069 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:08:14.458168 10116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.455500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:38869 (local address 127.1.244.193:53971)
0923 22:08:14.455679 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.455690 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.455741 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.455861 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.455861 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.455863 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:14.455941 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.455943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.456574 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.456575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.457151 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.457153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.457732 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.457733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.457781 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.457793 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.457986 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.457988 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:14.458041 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:08:14.458085 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260923 22:08:14.460202 9801 raft_consensus.cc:1261] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Refusing update from remote peer 7d3989ca76c34afc80f4bd1a98106943: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:14.460244 9941 raft_consensus.cc:1261] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Refusing update from remote peer 7d3989ca76c34afc80f4bd1a98106943: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:14.460491 10111 consensus_queue.cc:1035] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [LEADER]: Connected to new peer: Peer: permanent_uuid: "230612f73ece4f33aadd80548bfbaa84" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:14.460614 10023 consensus_queue.cc:1035] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f491a37f2184bcd83dba05de659aa75" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:14.467103 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.464986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:36841 (local address 127.1.244.193:41851)
0923 22:08:14.465022 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.465024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.465036 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:08:14.465076 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.465077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.465085 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.465149 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:14.465253 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.465696 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.466421 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.466668 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.466728 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.466867 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.466902 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.466952 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.467021 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.467042 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:14.467139 10080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.464926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41851 (local address 127.1.244.194:36841)
0923 22:08:14.464975 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.464986 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.465024 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.465153 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.465154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.465155 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:14.465194 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.465195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.465772 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.465773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.466323 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.466325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.466727 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.466727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.466785 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.466793 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:14.466966 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:14.466968 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:14.467016 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:08:14.467067 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:08:14.473687 9502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:14.473793 9502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:14.474422 9502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59a05404acc64309a8525f5865050e02: Sending DeleteTablet for 3 replicas of tablet bf0a3045c73b4197a0ac13cff755fed8
I20260923 22:08:14.474673 9641 tablet_service.cc:1514] Processing DeleteTablet for tablet bf0a3045c73b4197a0ac13cff755fed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:45700
I20260923 22:08:14.474759 9920 tablet_service.cc:1514] Processing DeleteTablet for tablet bf0a3045c73b4197a0ac13cff755fed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44578
I20260923 22:08:14.474854 10086 tablet_replica.cc:330] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943: stopping tablet replica
I20260923 22:08:14.474906 10086 raft_consensus.cc:2229] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:14.474980 10086 raft_consensus.cc:2258] T bf0a3045c73b4197a0ac13cff755fed8 P 7d3989ca76c34afc80f4bd1a98106943 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:14.475037 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 9576
I20260923 22:08:14.475072 10085 tablet_replica.cc:330] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: stopping tablet replica
I20260923 22:08:14.475128 10085 raft_consensus.cc:2229] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:14.475163 10085 raft_consensus.cc:2258] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:14.476298 9781 tablet_service.cc:1514] Processing DeleteTablet for tablet bf0a3045c73b4197a0ac13cff755fed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:14 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:39938
I20260923 22:08:14.476435 10084 tablet_replica.cc:330] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: stopping tablet replica
I20260923 22:08:14.476509 10084 raft_consensus.cc:2229] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:14.476557 10084 raft_consensus.cc:2258] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:14.476567 10085 ts_tablet_manager.cc:1905] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:14.476861 10084 ts_tablet_manager.cc:1905] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:14.478113 10085 ts_tablet_manager.cc:1918] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:14.478191 10085 log.cc:1196] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/bf0a3045c73b4197a0ac13cff755fed8
I20260923 22:08:14.478433 10085 ts_tablet_manager.cc:1939] T bf0a3045c73b4197a0ac13cff755fed8 P 9f491a37f2184bcd83dba05de659aa75: Deleting consensus metadata
I20260923 22:08:14.478617 10084 ts_tablet_manager.cc:1918] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:14.478682 10084 log.cc:1196] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/bf0a3045c73b4197a0ac13cff755fed8
I20260923 22:08:14.478837 9488 catalog_manager.cc:4841] TS 9f491a37f2184bcd83dba05de659aa75 (127.1.244.195:36567): tablet bf0a3045c73b4197a0ac13cff755fed8 (table test-table [id=aae682b5611e479581a3284b0536462f]) successfully deleted
I20260923 22:08:14.478879 10084 ts_tablet_manager.cc:1939] T bf0a3045c73b4197a0ac13cff755fed8 P 230612f73ece4f33aadd80548bfbaa84: Deleting consensus metadata
I20260923 22:08:14.479187 9491 catalog_manager.cc:4841] TS 230612f73ece4f33aadd80548bfbaa84 (127.1.244.194:38869): tablet bf0a3045c73b4197a0ac13cff755fed8 (table test-table [id=aae682b5611e479581a3284b0536462f]) successfully deleted
W20260923 22:08:14.482249 9490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:41851 (error 108)
I20260923 22:08:14.482515 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 9716
I20260923 22:08:14.483320 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.482605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41851 (local address 127.0.0.1:45724)
0923 22:08:14.483222 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.483273 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":59,"threads_started":1}
W20260923 22:08:14.483465 9490 catalog_manager.cc:4584] TS 7d3989ca76c34afc80f4bd1a98106943 (127.1.244.193:41851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf0a3045c73b4197a0ac13cff755fed8: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41851: connect: Connection refused (error 111)
I20260923 22:08:14.489576 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 9856
I20260923 22:08:14.495029 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 9472
2026-09-23T22:08:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2933 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:14 dist-test-slave-b5jz krb5kdc[10125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:14 dist-test-slave-b5jz krb5kdc[10125](info): set up 2 sockets
Sep 23 22:08:14 dist-test-slave-b5jz krb5kdc[10125](info): commencing operation
krb5kdc: starting...
W20260923 22:08:15.528290 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:08:15 dist-test-slave-b5jz krb5kdc[10125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:08:15.644277 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33839
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:33839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:15.724187 10138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:15.724386 10138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:15.724409 10138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:15.725550 10138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:15.725591 10138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:15.725605 10138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:15.725617 10138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:15.725629 10138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:15.727361 10138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:33839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:15.727648 10138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:15.727875 10138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:15.732615 10144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:15.732633 10146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:15.732775 10138 server_base.cc:1034] running on GCE node
W20260923 22:08:15.732673 10143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:15.733150 10138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:15.733440 10138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:15.734578 10138 hybrid_clock.cc:648] HybridClock initialized: now 1790201295734560 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-23T22:08:15Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1245 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:15 dist-test-slave-b5jz krb5kdc[10151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:15 dist-test-slave-b5jz krb5kdc[10151](info): set up 2 sockets
Sep 23 22:08:15 dist-test-slave-b5jz krb5kdc[10151](info): commencing operation
krb5kdc: starting...
W20260923 22:08:16.788214 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:08:16 dist-test-slave-b5jz krb5kdc[10151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:08:16.905690 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35397
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:16.987033 10164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:16.987250 10164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:16.987272 10164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:16.988457 10164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:16.988503 10164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:16.988518 10164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:16.988530 10164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:16.988543 10164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:16.989985 10164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:16.990206 10164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:16.990399 10164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:16.995283 10172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:16.995323 10170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:16.995278 10169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:16.995498 10164 server_base.cc:1034] running on GCE node
I20260923 22:08:16.995802 10164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:16.996054 10164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:16.997193 10164 hybrid_clock.cc:648] HybridClock initialized: now 1790201296997182 us; error 28 us; skew 500 ppm
Sep 23 22:08:16 dist-test-slave-b5jz krb5kdc[10151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201296, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:16.999620 10164 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:08:17.000072 10164 webserver.cc:460] Webserver started at http://127.1.244.254:45513/ using document root <none> and password file <none>
I20260923 22:08:17.000313 10164 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:17.000378 10164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:17.000528 10164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:17.001569 10164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "fb749559b52946fcb981a194e412d6fc"
format_stamp: "Formatted at 2026-09-23 22:08:16 on dist-test-slave-b5jz"
server_key: "7307441ccb7f05572e2464f35c6b6346"
server_key_iv: "4ae7cefec4600239708fe73a412e79d0"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.001927 10164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "fb749559b52946fcb981a194e412d6fc"
format_stamp: "Formatted at 2026-09-23 22:08:16 on dist-test-slave-b5jz"
server_key: "7307441ccb7f05572e2464f35c6b6346"
server_key_iv: "4ae7cefec4600239708fe73a412e79d0"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.003225 10164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:17.003988 10179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.004160 10164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.004257 10164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "fb749559b52946fcb981a194e412d6fc"
format_stamp: "Formatted at 2026-09-23 22:08:16 on dist-test-slave-b5jz"
server_key: "7307441ccb7f05572e2464f35c6b6346"
server_key_iv: "4ae7cefec4600239708fe73a412e79d0"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.004330 10164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:17.039832 10164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:17.041745 10164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:17.041916 10164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:17.046025 10164 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35397
I20260923 22:08:17.046064 10231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35397 every 8 connection(s)
I20260923 22:08:17.046444 10164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:17.047292 10232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:17.049573 10232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: Bootstrap starting.
I20260923 22:08:17.050103 10232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:17.050345 10232 log.cc:824] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:17.050642 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 10164
I20260923 22:08:17.050727 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:17.050952 2003 external_mini_cluster.cc:1428] Setting key 592d6e36e1552f7d040e4ed97641496c
I20260923 22:08:17.051108 10232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: No bootstrap required, opened a new log
I20260923 22:08:17.052888 10232 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } }
I20260923 22:08:17.053049 10232 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:17.053124 10232 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb749559b52946fcb981a194e412d6fc, State: Initialized, Role: FOLLOWER
I20260923 22:08:17.053296 10232 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } }
I20260923 22:08:17.053372 10232 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:17.053396 10232 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:17.053454 10232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:17.054247 10232 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } }
I20260923 22:08:17.054333 10232 leader_election.cc:302] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb749559b52946fcb981a194e412d6fc; no voters:
I20260923 22:08:17.054509 10232 leader_election.cc:288] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:17.054531 10237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:17.054642 10237 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 1 LEADER]: Becoming Leader. State: Replica: fb749559b52946fcb981a194e412d6fc, State: Running, Role: LEADER
I20260923 22:08:17.054723 10237 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } }
I20260923 22:08:17.054762 10232 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:17.055032 10237 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb749559b52946fcb981a194e412d6fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } } }
I20260923 22:08:17.055187 10237 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:17.055377 10237 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb749559b52946fcb981a194e412d6fc. Latest consensus state: current_term: 1 leader_uuid: "fb749559b52946fcb981a194e412d6fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } } }
I20260923 22:08:17.055524 10244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:17.055513 10237 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:17.056133 10244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:17.057772 10244 catalog_manager.cc:1347] Generated new cluster ID: f38aa323179f49c8aa1b9e81440e9df4
I20260923 22:08:17.057823 10244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:17.063963 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.051425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47580 (local address 127.1.244.254:35397)
0923 22:08:17.051618 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:17.051628 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:17.051672 (+ 44us) server_negotiation.cc:407] Connection header received
0923 22:08:17.052866 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:17.052872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:17.052926 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:17.053067 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:17.053844 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.055074 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.056242 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.056848 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.057262 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.059788 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:17.059792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:17.059792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:17.059804 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:17.061754 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.062158 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.062161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.062162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.062303 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.062510 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.062511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.062511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.062816 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:17.063321 (+ 505us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:17.063391 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:08:17.063521 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260923 22:08:17.065109 10244 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:17.065605 10244 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:17.068722 10244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: Generated new TSK 0
I20260923 22:08:17.068897 10244 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:08:17.096480 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35397
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:17.176177 10258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:17.176406 10258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:17.176445 10258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:17.177620 10258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:17.177685 10258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:17.177731 10258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:17.179147 10258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:17.179394 10258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:17.179658 10258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:17.184494 10264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:17.184479 10263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.184651 10258 server_base.cc:1034] running on GCE node
W20260923 22:08:17.184469 10266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.184959 10258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:17.185160 10258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:17.186296 10258 hybrid_clock.cc:648] HybridClock initialized: now 1790201297186287 us; error 27 us; skew 500 ppm
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:17.188732 10258 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:08:17.189122 10258 webserver.cc:460] Webserver started at http://127.1.244.193:46057/ using document root <none> and password file <none>
I20260923 22:08:17.189302 10258 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:17.189343 10258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:17.189428 10258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:17.190418 10258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "cf9622e9a3f242b39447b125cfddbf8b"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "7bec10318f70ac614007eda0067c3d1d"
server_key_iv: "148ba77baba36fe7934ee62ff8f26a9d"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.190734 10258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "cf9622e9a3f242b39447b125cfddbf8b"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "7bec10318f70ac614007eda0067c3d1d"
server_key_iv: "148ba77baba36fe7934ee62ff8f26a9d"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.192121 10258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:17.192775 10273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.192950 10258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:17.193019 10258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "cf9622e9a3f242b39447b125cfddbf8b"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "7bec10318f70ac614007eda0067c3d1d"
server_key_iv: "148ba77baba36fe7934ee62ff8f26a9d"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.193070 10258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:17.204746 10258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:17.206984 10258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:17.207140 10258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:17.207345 10258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:17.207691 10258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:17.207752 10258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.207782 10258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:17.207799 10258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.213462 10258 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:39625
I20260923 22:08:17.213514 10386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:39625 every 8 connection(s)
I20260923 22:08:17.213832 10258 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:17.221025 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 10258
I20260923 22:08:17.221184 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:17.221401 2003 external_mini_cluster.cc:1428] Setting key 51c63a1ba55a864b6a2dc78a2c561737
I20260923 22:08:17.222483 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.214512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:55539 (local address 127.1.244.254:35397)
0923 22:08:17.214571 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:17.214573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:17.214786 (+ 213us) server_negotiation.cc:407] Connection header received
0923 22:08:17.215255 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:17.215256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:17.215268 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:17.215311 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:17.216230 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.216724 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.217650 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.217846 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.217895 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.219836 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:17.219837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:17.219838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:17.219844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:17.221003 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.221500 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.221501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.221502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.221575 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.221857 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.221858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.221858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.221980 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:17.222048 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:17.222354 (+ 306us) server_negotiation.cc:299] Negotiation successful
0923 22:08:17.222422 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:08:17.222793 10389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.214494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:55539)
0923 22:08:17.214664 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:17.214705 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:17.215158 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:17.215363 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:17.215366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:17.215971 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:17.216153 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.216158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.216797 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.216797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.217572 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.217573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.217912 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.217913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.218009 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.218406 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:17.218414 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:17.219755 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:17.221081 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:17.221088 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:17.221090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:17.221399 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:17.221649 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:17.221649 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:17.221650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:17.221795 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:17.222047 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:17.222053 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:17.222212 (+ 159us) client_negotiation.cc:769] Sending connection context
0923 22:08:17.222334 (+ 122us) client_negotiation.cc:240] Negotiation successful
0923 22:08:17.222505 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260923 22:08:17.223796 10387 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35397
I20260923 22:08:17.223897 10387 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:17.224062 10387 heartbeater.cc:511] Master 127.1.244.254:35397 requested a full tablet report, sending...
I20260923 22:08:17.224607 10196 ts_manager.cc:194] Registered new tserver with Master: cf9622e9a3f242b39447b125cfddbf8b (127.1.244.193:39625)
I20260923 22:08:17.225193 10196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:55539
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:08:17.244536 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35397
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:17.325682 10392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:17.325886 10392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:17.325908 10392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:17.326984 10392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:17.327025 10392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:17.327051 10392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:17.328390 10392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:17.328612 10392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:17.328778 10392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:17.333389 10398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:17.333398 10400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:17.333398 10397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.333494 10392 server_base.cc:1034] running on GCE node
I20260923 22:08:17.333874 10392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:17.334131 10392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:17.335294 10392 hybrid_clock.cc:648] HybridClock initialized: now 1790201297335279 us; error 40 us; skew 500 ppm
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:17.337774 10392 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:08:17.338182 10392 webserver.cc:460] Webserver started at http://127.1.244.194:35497/ using document root <none> and password file <none>
I20260923 22:08:17.338395 10392 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:17.338459 10392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:17.338601 10392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:17.339659 10392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "3bd8751b7589455aba4fc09216a117ea"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "0985dedfdeb6624baa57488821b1d695"
server_key_iv: "12400be42e7153e358bc7e66301cd5e8"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.340003 10392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "3bd8751b7589455aba4fc09216a117ea"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "0985dedfdeb6624baa57488821b1d695"
server_key_iv: "12400be42e7153e358bc7e66301cd5e8"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.341274 10392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:17.341953 10407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.342135 10392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:17.342233 10392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "3bd8751b7589455aba4fc09216a117ea"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "0985dedfdeb6624baa57488821b1d695"
server_key_iv: "12400be42e7153e358bc7e66301cd5e8"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.342301 10392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:17.369701 10392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:17.371484 10392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:17.371658 10392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:17.371886 10392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:17.372187 10392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:17.372239 10392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.372279 10392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:17.372308 10392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.377838 10392 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:37463
I20260923 22:08:17.377884 10520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:37463 every 8 connection(s)
I20260923 22:08:17.378180 10392 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:17.379206 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 10392
I20260923 22:08:17.379312 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:17.379514 2003 external_mini_cluster.cc:1428] Setting key 23aff4f5f49c4861807d62a20b9bfcbf
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:17.388440 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.378844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:47697 (local address 127.1.244.254:35397)
0923 22:08:17.378887 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:17.378890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:17.379089 (+ 199us) server_negotiation.cc:407] Connection header received
0923 22:08:17.379802 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:17.379803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:17.379819 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:17.379882 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:17.381165 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.381958 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.383337 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.383528 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.383627 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.385736 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:17.385738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:17.385738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:17.385746 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:17.387080 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.387470 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.387472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.387472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.387535 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.387812 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.387813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.387813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.387924 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:17.387993 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:17.388263 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 22:08:17.388349 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:08:17.388636 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.378811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:47697)
0923 22:08:17.378974 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:17.379010 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:17.379640 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:17.379966 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:17.379971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:17.380777 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:17.381069 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.381076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.382041 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.382042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.383168 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.383171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.383644 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.383645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.383765 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.384297 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:17.384305 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:17.385630 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:17.387169 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:17.387175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:17.387178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:17.387396 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:17.387628 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:17.387629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:17.387629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:17.387741 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:17.387994 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:17.388001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:17.388136 (+ 135us) client_negotiation.cc:769] Sending connection context
0923 22:08:17.388241 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:08:17.388394 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
I20260923 22:08:17.389096 10521 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35397
I20260923 22:08:17.389206 10521 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:17.389386 10521 heartbeater.cc:511] Master 127.1.244.254:35397 requested a full tablet report, sending...
I20260923 22:08:17.389775 10196 ts_manager.cc:194] Registered new tserver with Master: 3bd8751b7589455aba4fc09216a117ea (127.1.244.194:37463)
I20260923 22:08:17.390174 10196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:47697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:08:17.404656 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35397
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:17.483129 10526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:17.483320 10526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:17.483345 10526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:17.484516 10526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:17.484566 10526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:17.484620 10526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:17.486001 10526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:17.486238 10526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:17.486471 10526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:17.491310 10526 server_base.cc:1034] running on GCE node
W20260923 22:08:17.491283 10534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:17.491304 10531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:17.491281 10532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.491757 10526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:17.491977 10526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:17.493101 10526 hybrid_clock.cc:648] HybridClock initialized: now 1790201297493090 us; error 29 us; skew 500 ppm
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:17.495517 10526 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:08:17.496073 10526 webserver.cc:460] Webserver started at http://127.1.244.195:35045/ using document root <none> and password file <none>
I20260923 22:08:17.496292 10526 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:17.496354 10526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:17.496484 10526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:17.497473 10526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "852c9cccb80d4d0c80a4d25b22dfe6bd"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "fc2a3344f6f5f5c9ff78866ae6d366b7"
server_key_iv: "91a8e136568228ff1df981d3f79b230a"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.497839 10526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "852c9cccb80d4d0c80a4d25b22dfe6bd"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "fc2a3344f6f5f5c9ff78866ae6d366b7"
server_key_iv: "91a8e136568228ff1df981d3f79b230a"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.499127 10526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:17.499857 10541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.500036 10526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.500131 10526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "852c9cccb80d4d0c80a4d25b22dfe6bd"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "fc2a3344f6f5f5c9ff78866ae6d366b7"
server_key_iv: "91a8e136568228ff1df981d3f79b230a"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.500212 10526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:17.521301 10526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:17.523618 10526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:17.523767 10526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:17.523988 10526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:17.524286 10526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:17.524334 10526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.524374 10526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:17.524405 10526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.530093 10526 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:37283
I20260923 22:08:17.530164 10654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:37283 every 8 connection(s)
I20260923 22:08:17.530457 10526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:17.538625 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.531111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:51549 (local address 127.1.244.254:35397)
0923 22:08:17.531163 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:17.531166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:17.531378 (+ 212us) server_negotiation.cc:407] Connection header received
0923 22:08:17.531844 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:17.531845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:17.531855 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:17.531907 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:17.532749 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.533226 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.534125 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.534298 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.534346 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.536378 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:17.536380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:17.536380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:17.536387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:17.537530 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.537867 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.537868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.537868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.537917 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.538115 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.538115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.538116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.538208 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:17.538257 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:17.538498 (+ 241us) server_negotiation.cc:299] Negotiation successful
0923 22:08:17.538557 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:17.538784 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.531108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:51549)
0923 22:08:17.531267 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:17.531301 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:17.531765 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:17.531956 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:17.531959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:17.532492 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:17.532682 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.532686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.533295 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.533295 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.534057 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.534059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.534361 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.534362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.534475 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.534985 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:17.534999 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:17.536297 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:17.537596 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:17.537601 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:17.537603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:17.537805 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:17.537971 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:17.537971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:17.537971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:17.538066 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:17.538258 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:17.538261 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:17.538362 (+ 101us) client_negotiation.cc:769] Sending connection context
0923 22:08:17.538456 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:08:17.538575 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260923 22:08:17.539261 10655 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35397
I20260923 22:08:17.539311 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 10526
I20260923 22:08:17.539358 10655 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:17.539423 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:17.539592 10655 heartbeater.cc:511] Master 127.1.244.254:35397 requested a full tablet report, sending...
I20260923 22:08:17.539676 2003 external_mini_cluster.cc:1428] Setting key d600196edcdfdfe3d552ac40ccf94c9d
I20260923 22:08:17.539985 10196 ts_manager.cc:194] Registered new tserver with Master: 852c9cccb80d4d0c80a4d25b22dfe6bd (127.1.244.195:37283)
I20260923 22:08:17.540426 10196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:51549
I20260923 22:08:17.540553 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
WARNING: no policy specified for oryx/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
WARNING: no policy specified for oryx/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
I20260923 22:08:17.608124 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 10258
I20260923 22:08:17.612512 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 10392
I20260923 22:08:17.616607 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 10526
I20260923 22:08:17.620672 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 10164
I20260923 22:08:17.625694 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35397
--webserver_interface=127.1.244.254
--webserver_port=45513
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:17.705435 10664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:17.705665 10664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:17.705701 10664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:17.706864 10664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:17.706969 10664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:17.706996 10664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:17.707038 10664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:17.707062 10664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:17.708544 10664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=45513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:17.708801 10664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:17.709023 10664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:17.713858 10670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.713974 10664 server_base.cc:1034] running on GCE node
W20260923 22:08:17.713855 10669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:17.713853 10672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.714380 10664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:17.714619 10664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:17.715758 10664 hybrid_clock.cc:648] HybridClock initialized: now 1790201297715747 us; error 27 us; skew 500 ppm
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:17.718091 10664 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:08:17.718525 10664 webserver.cc:460] Webserver started at http://127.1.244.254:45513/ using document root <none> and password file <none>
I20260923 22:08:17.718734 10664 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:17.718804 10664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:17.720167 10664 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:17.720782 10679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.721025 10664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:17.721139 10664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "fb749559b52946fcb981a194e412d6fc"
format_stamp: "Formatted at 2026-09-23 22:08:16 on dist-test-slave-b5jz"
server_key: "7307441ccb7f05572e2464f35c6b6346"
server_key_iv: "4ae7cefec4600239708fe73a412e79d0"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.721464 10664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:17.743928 10664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:17.745734 10664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:17.745893 10664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:17.749928 10664 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35397
I20260923 22:08:17.749995 10731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35397 every 8 connection(s)
I20260923 22:08:17.750370 10664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:17.751787 10732 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:08:17.752429 10732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: Bootstrap starting.
I20260923 22:08:17.760327 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 10664
I20260923 22:08:17.760440 10732 log.cc:824] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:17.760641 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:39625
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:35397
--webserver_port=46057
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.244.193
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:08:17.762432 10732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:08:17.762706 10732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: Bootstrap complete.
I20260923 22:08:17.765004 10732 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } }
I20260923 22:08:17.765221 10732 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb749559b52946fcb981a194e412d6fc, State: Initialized, Role: FOLLOWER
I20260923 22:08:17.765374 10732 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } }
I20260923 22:08:17.765442 10732 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:17.765486 10732 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:17.765532 10732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:08:17.766474 10732 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } }
I20260923 22:08:17.766583 10732 leader_election.cc:302] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb749559b52946fcb981a194e412d6fc; no voters:
I20260923 22:08:17.766764 10732 leader_election.cc:288] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:08:17.766798 10736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:08:17.766914 10736 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [term 2 LEADER]: Becoming Leader. State: Replica: fb749559b52946fcb981a194e412d6fc, State: Running, Role: LEADER
I20260923 22:08:17.767021 10732 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:17.767109 10736 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } }
I20260923 22:08:17.767455 10737 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fb749559b52946fcb981a194e412d6fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } } }
I20260923 22:08:17.767562 10737 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:17.767925 10738 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb749559b52946fcb981a194e412d6fc. Latest consensus state: current_term: 2 leader_uuid: "fb749559b52946fcb981a194e412d6fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb749559b52946fcb981a194e412d6fc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35397 } } }
I20260923 22:08:17.768003 10738 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:17.768149 10743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:17.768684 10743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:17.768807 10743 catalog_manager.cc:1259] Loaded cluster ID: f38aa323179f49c8aa1b9e81440e9df4
I20260923 22:08:17.768841 10743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:17.769997 10743 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:17.770172 10743 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fb749559b52946fcb981a194e412d6fc: Loaded TSK: 0
I20260923 22:08:17.770380 10743 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:08:17.845903 10735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:17.846127 10735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:17.846169 10735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:17.847321 10735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:17.847385 10735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:17.847431 10735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:17.848886 10735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:39625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=46057
--tserver_master_addrs=127.1.244.254:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:17.849136 10735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:17.849354 10735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:17.855165 10761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:17.855186 10759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.855311 10735 server_base.cc:1034] running on GCE node
W20260923 22:08:17.855181 10758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.855669 10735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:17.855907 10735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:17.857064 10735 hybrid_clock.cc:648] HybridClock initialized: now 1790201297857054 us; error 33 us; skew 500 ppm
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:17.867123 10735 init.cc:377] Logged in from keytab as oryx/127.1.244.193@KRBTEST.COM (short username oryx)
I20260923 22:08:17.867682 10735 webserver.cc:460] Webserver started at http://127.1.244.193:46057/ using document root <none> and password file <none>
I20260923 22:08:17.867941 10735 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:17.868016 10735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:17.869683 10735 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:17.870303 10768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.870509 10735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:17.870603 10735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "cf9622e9a3f242b39447b125cfddbf8b"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "7bec10318f70ac614007eda0067c3d1d"
server_key_iv: "148ba77baba36fe7934ee62ff8f26a9d"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.871024 10735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:17.887418 10735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:17.889686 10735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:17.889833 10735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:17.890053 10735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:17.890337 10735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:17.890386 10735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.890415 10735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:17.890440 10735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.897739 10735 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:39625
I20260923 22:08:17.897805 10881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:39625 every 8 connection(s)
I20260923 22:08:17.898180 10735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:17.905994 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 10735
I20260923 22:08:17.906296 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:37463
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:35397
--webserver_port=35497
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.244.194
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:08:17.906414 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.898890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:48773)
0923 22:08:17.899056 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:17.899097 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:17.899592 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:17.900047 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:17.900050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:17.900617 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:17.900792 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.900797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.901893 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.901894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.902667 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.902669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.903255 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.903256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.903383 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.903883 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:17.903891 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:17.906207 (+ 2316us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":102,"threads_started":1}
W20260923 22:08:17.906600 10882 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:35397 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.244.254:35397: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
W20260923 22:08:17.906736 10885 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:17.898968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:48773 (local address 127.1.244.254:35397)
0923 22:08:17.899685 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:17.899694 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:17.899730 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:08:17.899838 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:17.899842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:17.899880 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:17.900019 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:17.900914 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.901793 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.902758 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.903169 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.903282 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.906337 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:48773: BlockingRecv error: recv got EOF from 127.1.244.193:48773 (error 108)
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":94,"threads_started":1}
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:17.912231 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.906815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:49923)
0923 22:08:17.906895 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:17.906906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:17.906946 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:17.907079 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:17.907080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:17.907202 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:17.907242 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.907244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.908183 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.908185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.908928 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.908931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.909442 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.909444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.909512 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.909974 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:17.909980 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:17.912162 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260923 22:08:17.912354 10885 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:17.906885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:49923 (local address 127.1.244.254:35397)
0923 22:08:17.906934 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:17.906936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:17.906944 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:17.907007 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:17.907008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:17.907020 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:17.907071 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:17.907314 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.908051 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.909038 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.909363 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.909430 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.912270 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:49923: BlockingRecv error: recv got EOF from 127.1.244.193:49923 (error 108)
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":34}
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:17.917975 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.912708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:52119)
0923 22:08:17.912758 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:17.912770 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:17.912814 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:17.913067 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:17.913069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:17.913222 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:17.913258 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.913259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.914142 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.914143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.915019 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.915022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.915485 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.915486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.915571 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.915985 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:17.915992 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:17.917913 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:08:17.918100 10882 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:08:17.918222 10887 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:17.912807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:52119 (local address 127.1.244.254:35397)
0923 22:08:17.912940 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:17.912941 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:17.912948 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:17.912967 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:17.912967 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:17.912978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:17.913046 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:17.913341 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.914061 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.915114 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.915413 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.915470 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.918008 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:52119: BlockingRecv error: recv got EOF from 127.1.244.193:52119 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
W20260923 22:08:17.989677 10886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:17.989918 10886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:17.989957 10886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:17.991132 10886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:17.991200 10886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:17.991243 10886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:17.992722 10886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:37463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=35497
--tserver_master_addrs=127.1.244.254:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:17.992982 10886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:17.993203 10886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:17.998054 10892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.998119 10886 server_base.cc:1034] running on GCE node
W20260923 22:08:17.998059 10895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:17.998060 10893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.998507 10886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:17.998734 10886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:17.999863 10886 hybrid_clock.cc:648] HybridClock initialized: now 1790201297999844 us; error 36 us; skew 500 ppm
Sep 23 22:08:17 dist-test-slave-b5jz krb5kdc[10151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:18.002575 10886 init.cc:377] Logged in from keytab as oryx/127.1.244.194@KRBTEST.COM (short username oryx)
I20260923 22:08:18.003023 10886 webserver.cc:460] Webserver started at http://127.1.244.194:35497/ using document root <none> and password file <none>
I20260923 22:08:18.003237 10886 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:18.003307 10886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:18.004644 10886 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:18.005192 10902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.005362 10886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:18.005450 10886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "3bd8751b7589455aba4fc09216a117ea"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "0985dedfdeb6624baa57488821b1d695"
server_key_iv: "12400be42e7153e358bc7e66301cd5e8"
server_key_version: "encryptionkey@0"
I20260923 22:08:18.005765 10886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:18.035729 10886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:18.037573 10886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:18.037724 10886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:18.037947 10886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:18.038250 10886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:18.038283 10886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.038322 10886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:18.038349 10886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.043963 10886 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:37463
I20260923 22:08:18.044004 11015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:37463 every 8 connection(s)
I20260923 22:08:18.044339 10886 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:08:18.051252 10887 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:18.045052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:41589 (local address 127.1.244.254:35397)
0923 22:08:18.045131 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:18.045133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:18.045293 (+ 160us) server_negotiation.cc:407] Connection header received
0923 22:08:18.045765 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:18.045766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:18.045778 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:18.045829 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:18.046726 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.047216 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.048160 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.048345 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.048393 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.051207 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:41589: BlockingRecv error: recv got EOF from 127.1.244.194:41589 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:08:18.051268 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.045013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:41589)
0923 22:08:18.045179 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:18.045216 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:18.045685 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:18.045890 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:18.045893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:18.046453 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:18.046652 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.046656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.047289 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.047290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.048084 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.048087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.048408 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.048409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.048510 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.049006 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:18.049019 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:18.051131 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
W20260923 22:08:18.051493 11016 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:35397 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.244.254:35397: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
I20260923 22:08:18.051978 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 10886
I20260923 22:08:18.052260 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:37283
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:35397
--webserver_port=35045
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.244.195
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:18.055864 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.051725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:55717)
0923 22:08:18.051771 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:18.051782 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:18.051856 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:18.051972 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:18.051973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:18.052116 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:18.052162 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.052163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.052846 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.052847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.053341 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.053343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.053704 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.053704 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.053769 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.054189 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:18.054194 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:18.055782 (+ 1588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:08:18.055915 10887 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:18.051807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:55717 (local address 127.1.244.254:35397)
0923 22:08:18.051870 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:18.051872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:18.051880 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:18.051904 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:18.051905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:18.051916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:18.051960 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:18.052230 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.052771 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.053419 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.053623 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.053703 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.055850 (+ 2147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:55717: BlockingRecv error: recv got EOF from 127.1.244.194:55717 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:18.059963 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.056102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:52001)
0923 22:08:18.056211 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:18.056220 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:18.056262 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:18.056385 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:18.056386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:18.056511 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:18.056534 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.056535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.057136 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.057137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.057609 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.057610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.057934 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.057935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.057985 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.058361 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:18.058366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:18.059885 (+ 1519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
W20260923 22:08:18.060020 10887 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:18.056142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:52001 (local address 127.1.244.254:35397)
0923 22:08:18.056179 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:18.056181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:18.056249 (+ 68us) server_negotiation.cc:407] Connection header received
0923 22:08:18.056318 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:18.056320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:18.056330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:18.056373 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:18.056601 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.057051 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.057683 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.057867 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.057936 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.059951 (+ 2015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:52001: BlockingRecv error: recv got EOF from 127.1.244.194:52001 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:08:18.060073 11016 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:08:18.133113 11019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:18.133344 11019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:18.133383 11019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:18.134542 11019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:18.134603 11019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:18.134647 11019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:18.136209 11019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:37283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=35045
--tserver_master_addrs=127.1.244.254:35397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:18.136469 11019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:18.136694 11019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:18.141377 11028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:18.141413 11026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:18.141382 11025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:18.141635 11019 server_base.cc:1034] running on GCE node
I20260923 22:08:18.141835 11019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:18.142071 11019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:18.143231 11019 hybrid_clock.cc:648] HybridClock initialized: now 1790201298143218 us; error 33 us; skew 500 ppm
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201298, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:18.145704 11019 init.cc:377] Logged in from keytab as oryx/127.1.244.195@KRBTEST.COM (short username oryx)
I20260923 22:08:18.146091 11019 webserver.cc:460] Webserver started at http://127.1.244.195:35045/ using document root <none> and password file <none>
I20260923 22:08:18.146265 11019 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:18.146338 11019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:18.147635 11019 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:18.148161 11035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.148319 11019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.148403 11019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "852c9cccb80d4d0c80a4d25b22dfe6bd"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-b5jz"
server_key: "fc2a3344f6f5f5c9ff78866ae6d366b7"
server_key_iv: "91a8e136568228ff1df981d3f79b230a"
server_key_version: "encryptionkey@0"
I20260923 22:08:18.148773 11019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:18.170233 11019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:18.172487 11019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:18.172595 11019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:18.172799 11019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:18.173099 11019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:18.173151 11019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.173190 11019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:18.173219 11019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.178751 11019 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:37283
I20260923 22:08:18.178791 11148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:37283 every 8 connection(s)
I20260923 22:08:18.179129 11019 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:18.185817 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.179801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:54375)
0923 22:08:18.179939 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:18.179977 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:18.180477 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:18.180691 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:18.180694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:18.181223 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:18.181401 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.181405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.182029 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.182029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.182797 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.182799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.183118 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.183119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.183220 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.183717 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:18.183725 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:18.185686 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
W20260923 22:08:18.185825 10887 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:18.179823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:54375 (local address 127.1.244.254:35397)
0923 22:08:18.179867 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:18.179868 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:18.180082 (+ 214us) server_negotiation.cc:407] Connection header received
0923 22:08:18.180556 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:18.180557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:18.180566 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:18.180641 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:18.181481 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.181957 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.182868 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.183036 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.183101 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.185775 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:54375: BlockingRecv error: recv got EOF from 127.1.244.195:54375 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 22:08:18.186095 11149 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:35397 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.244.254:35397: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
I20260923 22:08:18.186955 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 11019
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:18.190608 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.186301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:42595)
0923 22:08:18.186345 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:18.186355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:18.186391 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:18.186585 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:18.186585 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:18.186686 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:18.186725 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.186726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.187530 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.187530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.188044 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.188046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.188494 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.188495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.188559 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.188916 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:18.188920 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:18.190566 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:08:18.190719 10887 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:18.186448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:42595 (local address 127.1.244.254:35397)
0923 22:08:18.186486 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:18.186488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:18.186495 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:18.186518 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:18.186519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:18.186526 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:18.186565 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:18.186791 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.187444 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.188120 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.188410 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.188479 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.190643 (+ 2164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:42595: BlockingRecv error: recv got EOF from 127.1.244.195:42595 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:18.194579 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.187347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47592 (local address 127.1.244.254:35397)
0923 22:08:18.187394 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:18.187396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:18.187576 (+ 180us) server_negotiation.cc:407] Connection header received
0923 22:08:18.187659 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:18.187660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:18.187672 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:18.187713 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:18.188341 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.188996 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.189746 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.189933 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.190005 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.191432 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:18.191436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:18.191437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:18.191448 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:18.193166 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:18.193541 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:18.193544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:18.193545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:18.193637 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:18.193848 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:18.193849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:18.193849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:18.194107 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:18.194248 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:18.194357 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 22:08:18.194465 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:18.194976 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.190863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:44365)
0923 22:08:18.190895 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:18.190905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:18.190940 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:18.191088 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:18.191088 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:18.191205 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:18.191227 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.191228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.192027 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.192028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.192593 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.192595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.193000 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.193001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.193034 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.193364 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:18.193368 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:18.194909 (+ 1541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260923 22:08:18.195060 10887 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:18.190941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:44365 (local address 127.1.244.254:35397)
0923 22:08:18.190988 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:18.190990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:18.190996 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:08:18.191012 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:18.191013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:18.191020 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:18.191086 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:18.191299 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.191956 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.192665 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.192927 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.193001 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.195011 (+ 2010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:44365: BlockingRecv error: recv got EOF from 127.1.244.195:44365 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260923 22:08:18.195106 11149 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:18 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:18.923151 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.918422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:53623)
0923 22:08:18.918554 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:18.918565 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:18.918613 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:18.918727 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:18.918728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:18.918877 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:18.918931 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.918933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.919703 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.919705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.920568 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.920571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.920890 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.920891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.920932 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.921330 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:18.921334 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:18.923044 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260923 22:08:18.923199 11154 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:18.918494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53623 (local address 127.1.244.254:35397)
0923 22:08:18.918602 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:18.918603 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:18.918611 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:18.918656 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:18.918657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:18.918666 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:18.918714 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:18.918985 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.919608 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.920640 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.920834 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.920889 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.923123 (+ 2234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:53623: BlockingRecv error: recv got EOF from 127.1.244.193:53623 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":61,"threads_started":1}
Sep 23 22:08:19 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:19 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:19.064769 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:19.060448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:35805)
0923 22:08:19.060581 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:19.060592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:19.060649 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:19.060781 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:19.060782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:19.060944 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:19.060992 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:19.060993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.061612 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.061612 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:19.062174 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:19.062175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.062480 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.062480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:19.062518 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:19.062941 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:19.062945 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:19.064700 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
W20260923 22:08:19.064895 11154 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:19.060504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:35805 (local address 127.1.244.254:35397)
0923 22:08:19.060564 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:19.060571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:19.060644 (+ 73us) server_negotiation.cc:407] Connection header received
0923 22:08:19.060711 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:19.060712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:19.060723 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:19.060764 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:19.061047 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.061550 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.062240 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.062424 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.062471 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:19.064789 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:35805: BlockingRecv error: recv got EOF from 127.1.244.194:35805 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 23 22:08:19 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:19 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:19.199841 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:19.195457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:50193)
0923 22:08:19.195608 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:19.195620 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:19.195696 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:19.195844 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:19.195844 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:19.195998 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:19.196045 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:19.196047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.196682 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.196683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:19.197235 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:19.197237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.197545 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.197546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:19.197585 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:19.198003 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:19.198008 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:19.199775 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":50,"threads_started":1}
W20260923 22:08:19.199929 11154 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:19.195573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:50193 (local address 127.1.244.254:35397)
0923 22:08:19.195633 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:19.195636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:19.195686 (+ 50us) server_negotiation.cc:407] Connection header received
0923 22:08:19.195744 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:19.195745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:19.195759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:19.195815 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:19.196115 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.196603 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.197306 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.197487 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.197537 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:19.199857 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:50193: BlockingRecv error: recv got EOF from 127.1.244.195:50193 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 23 22:08:19 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:19 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:19.928687 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:19.923842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:47337)
0923 22:08:19.923998 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:19.924010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:19.924076 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:19.924205 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:19.924206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:19.924381 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:19.924432 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:19.924434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.925179 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.925179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:19.925919 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:19.925923 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.926289 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.926289 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:19.926328 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:19.926825 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:19.926829 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:19.928628 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
W20260923 22:08:19.928923 11158 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:19.923900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:47337 (local address 127.1.244.254:35397)
0923 22:08:19.924004 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:19.924007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:19.924080 (+ 73us) server_negotiation.cc:407] Connection header received
0923 22:08:19.924132 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:19.924132 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:19.924142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:19.924193 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:19.924496 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.925114 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.926012 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.926224 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.926281 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:19.928849 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:47337: BlockingRecv error: recv got EOF from 127.1.244.193:47337 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":48,"threads_started":1}
Sep 23 22:08:20 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:20 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:20.069654 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:20.065178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:49611)
0923 22:08:20.065364 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:20.065373 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:20.065423 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:20.065553 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:20.065554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:20.065717 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:20.065768 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:20.065769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.066379 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.066379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:20.067038 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:20.067040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.067344 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.067344 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:20.067382 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:20.067892 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:20.067897 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:20.069595 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
W20260923 22:08:20.069751 11158 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:20.065265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:49611 (local address 127.1.244.254:35397)
0923 22:08:20.065337 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:20.065339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:20.065424 (+ 85us) server_negotiation.cc:407] Connection header received
0923 22:08:20.065480 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:20.065481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:20.065491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:20.065533 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:20.065826 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.066316 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.067101 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.067282 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.067335 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:20.069685 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:49611: BlockingRecv error: recv got EOF from 127.1.244.194:49611 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 22:08:20 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:20 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:20.204907 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:20.200284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:59831)
0923 22:08:20.200430 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:20.200444 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:20.200510 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:20.200670 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:20.200671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:20.200848 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:20.200893 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:20.200895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.201554 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.201554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:20.202076 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:20.202077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.202438 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.202438 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:20.202499 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:20.202963 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:20.202968 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:20.204811 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
W20260923 22:08:20.204953 11158 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:20.200393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:59831 (local address 127.1.244.254:35397)
0923 22:08:20.200456 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:20.200457 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:20.200491 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:08:20.200578 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:20.200579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:20.200595 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:20.200663 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:20.200969 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.201461 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.202157 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.202350 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.202434 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:20.204888 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:59831: BlockingRecv error: recv got EOF from 127.1.244.195:59831 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 23 22:08:20 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:20 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:20.933467 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:20.929118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:54749)
0923 22:08:20.929254 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:20.929270 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:20.929326 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:20.929544 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:20.929545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:20.929742 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:20.929790 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:20.929792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.930416 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.930416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:20.930947 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:20.930949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.931269 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.931269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:20.931308 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:20.931773 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:20.931777 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:20.933410 (+ 1633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
W20260923 22:08:20.933604 11162 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:20.929232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:54749 (local address 127.1.244.254:35397)
0923 22:08:20.929365 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:20.929367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:20.929377 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:20.929402 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:20.929403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:20.929414 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:20.929511 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:20.929856 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.930351 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.931016 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.931204 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.931260 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:20.933512 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:54749: BlockingRecv error: recv got EOF from 127.1.244.193:54749 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":87,"threads_started":1}
Sep 23 22:08:21 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:21 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:21.074271 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:21.070106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:55407)
0923 22:08:21.070264 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:21.070274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:21.070326 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:21.070437 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:21.070438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:21.070593 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:21.070648 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.070650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.071293 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.071294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.071844 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.071845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.072151 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.072151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.072189 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.072591 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:21.072596 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:21.074215 (+ 1619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
W20260923 22:08:21.074368 11162 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:21.070174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:55407 (local address 127.1.244.254:35397)
0923 22:08:21.070232 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:21.070234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:21.070312 (+ 78us) server_negotiation.cc:407] Connection header received
0923 22:08:21.070371 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:21.070372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:21.070382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:21.070431 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:21.070701 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.071237 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.071909 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.072089 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.072145 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.074301 (+ 2156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:55407: BlockingRecv error: recv got EOF from 127.1.244.194:55407 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 22:08:21 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:21 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:21.210005 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:21.205377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:50875)
0923 22:08:21.205616 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:21.205626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:21.205683 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:21.205797 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:21.205798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:21.205977 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:21.206038 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.206045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.206841 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.206842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.207359 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.207361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.207753 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.207754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.207796 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.208204 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:21.208208 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:21.209946 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":175,"threads_started":1}
W20260923 22:08:21.210153 11162 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:21.205437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:50875 (local address 127.1.244.254:35397)
0923 22:08:21.205502 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:21.205504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:21.205679 (+ 175us) server_negotiation.cc:407] Connection header received
0923 22:08:21.205726 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:21.205727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:21.205738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:21.205791 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:21.206138 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.206774 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.207424 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.207663 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.207757 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.210028 (+ 2271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:50875: BlockingRecv error: recv got EOF from 127.1.244.195:50875 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 22:08:21 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:21 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:21.938411 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:21.933878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:50521)
0923 22:08:21.934050 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:21.934063 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:21.934126 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:21.934311 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:21.934311 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:21.934465 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:21.934524 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.934526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.935411 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.935412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.935959 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.935960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.936261 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.936262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.936308 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.936724 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:21.936729 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:21.938354 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
W20260923 22:08:21.938491 11166 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:21.934082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:50521 (local address 127.1.244.254:35397)
0923 22:08:21.934184 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:21.934186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:21.934194 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:21.934217 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:21.934218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:21.934234 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:21.934292 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:21.934583 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.935345 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.936016 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.936203 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.936254 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.938438 (+ 2184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:50521: BlockingRecv error: recv got EOF from 127.1.244.193:50521 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":57,"threads_started":1}
Sep 23 22:08:22 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:22 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:22.079070 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.074799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:43243)
0923 22:08:22.074943 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.074958 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.075011 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.075113 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.075114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.075270 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.075318 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.075319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.075984 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.075985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.076514 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.076516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.076825 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.076826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.076863 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.077282 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.077286 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.079012 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
W20260923 22:08:22.079183 11166 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:22.074867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:43243 (local address 127.1.244.254:35397)
0923 22:08:22.074914 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.074916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.075019 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:08:22.075048 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.075049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.075059 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.075109 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.075372 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.075917 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.076584 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.076766 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.076818 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.079107 (+ 2289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:43243: BlockingRecv error: recv got EOF from 127.1.244.194:43243 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 22:08:22 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:22 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:22.215458 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.210434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:34711)
0923 22:08:22.210595 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.210611 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.210694 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.210860 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.210861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.211049 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.211096 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.211098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.211811 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.211812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.212340 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.212342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.212826 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.212828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.212915 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.213501 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.213507 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.215400 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
W20260923 22:08:22.215606 11166 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:22.210498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:34711 (local address 127.1.244.254:35397)
0923 22:08:22.210578 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.210580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.210674 (+ 94us) server_negotiation.cc:407] Connection header received
0923 22:08:22.210774 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.210775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.210794 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.210852 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.211174 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.211733 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.212423 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.212738 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.212807 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.215502 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:34711: BlockingRecv error: recv got EOF from 127.1.244.195:34711 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 23 22:08:22 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:22 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:22.944764 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.938898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:41365)
0923 22:08:22.939073 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.939083 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.939139 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.939348 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.939349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.939568 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.939629 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.939632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.940633 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.940634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.941530 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.941532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.942091 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.942092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.942175 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.942690 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.942695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.944668 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":52,"threads_started":1}
W20260923 22:08:22.944859 11170 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:22.939064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:41365 (local address 127.1.244.254:35397)
0923 22:08:22.939192 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.939194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.939203 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:22.939227 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.939228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.939253 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.939330 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.939724 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.940541 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.941673 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.942017 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.942093 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.944762 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:41365: BlockingRecv error: recv got EOF from 127.1.244.193:41365 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":51,"threads_started":1}
Sep 23 22:08:23 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:23 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:23.083997 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.079516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:52051)
0923 22:08:23.079678 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:23.079687 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:23.079741 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:23.079875 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:23.079876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:23.080034 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:23.080082 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.080083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.080720 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.080720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.081273 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.081275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.081754 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.081755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.081817 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.082240 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:23.082244 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:23.083941 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
W20260923 22:08:23.084086 11170 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:23.079637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:52051 (local address 127.1.244.254:35397)
0923 22:08:23.079702 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.079704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.079728 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:08:23.079794 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.079794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.079804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.079854 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:23.080140 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.080656 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.081348 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.081680 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.081742 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.084024 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:52051: BlockingRecv error: recv got EOF from 127.1.244.194:52051 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 23 22:08:23 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:23 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:23.220404 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.216069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:57425)
0923 22:08:23.216202 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:23.216213 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:23.216262 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:23.216403 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:23.216404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:23.216563 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:23.216612 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.216613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.217246 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.217246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.217768 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.217769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.218085 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.218086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.218124 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.218559 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:23.218563 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:23.220345 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
W20260923 22:08:23.220546 11170 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:23.216148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:57425 (local address 127.1.244.254:35397)
0923 22:08:23.216234 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.216236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.216248 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:08:23.216326 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.216327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.216342 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.216385 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:23.216669 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.217182 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.217843 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.218022 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.218072 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.220453 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:57425: BlockingRecv error: recv got EOF from 127.1.244.195:57425 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 23 22:08:23 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:23 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:23.949749 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.945296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:57539)
0923 22:08:23.945414 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:23.945426 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:23.945482 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:23.945609 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:23.945611 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:23.945765 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:23.945815 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.945816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.946449 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.946450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.946978 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.946979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.947330 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.947331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.947397 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.947901 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:23.947907 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:23.949639 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
W20260923 22:08:23.949795 11174 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:23.945361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:57539 (local address 127.1.244.254:35397)
0923 22:08:23.945483 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.945484 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.945493 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:23.945519 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.945519 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.945529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.945597 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:23.945879 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.946381 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.947047 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.947233 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.947324 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.949722 (+ 2398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:57539: BlockingRecv error: recv got EOF from 127.1.244.193:57539 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
Sep 23 22:08:24 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:24 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:24.088979 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.084586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:35557)
0923 22:08:24.084733 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.084743 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:24.084800 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:24.084946 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:24.084946 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:24.085104 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:24.085151 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.085152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.085767 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.085768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.086279 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.086280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.086572 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.086573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.086610 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.087145 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:24.087150 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:24.088922 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
W20260923 22:08:24.089062 11174 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:24.084649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:35557 (local address 127.1.244.254:35397)
0923 22:08:24.084715 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.084720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.084785 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:08:24.084870 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.084871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.084883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.084930 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:24.085208 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.085707 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.086343 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.086517 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.086565 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.088995 (+ 2430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:35557: BlockingRecv error: recv got EOF from 127.1.244.194:35557 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 23 22:08:24 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:24 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:24.225893 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.220854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:37357)
0923 22:08:24.220974 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.220986 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:24.221044 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:24.221402 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:24.221404 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:24.221571 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:24.221620 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.221622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.222538 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.222538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.223057 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.223059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.223403 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.223404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.223447 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.223966 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:24.223971 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:24.225836 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
W20260923 22:08:24.226018 11174 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:24.220945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:37357 (local address 127.1.244.254:35397)
0923 22:08:24.221001 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.221003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.221030 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:08:24.221118 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.221119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.221129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.221191 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:24.221751 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.222469 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.223130 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.223312 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.223405 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.225921 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:37357: BlockingRecv error: recv got EOF from 127.1.244.195:37357 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 23 22:08:24 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:24 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:24.954869 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.950249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:55475)
0923 22:08:24.950405 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.950418 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:24.950485 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:24.950636 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:24.950638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:24.950847 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:24.950895 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.950897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.951585 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.951586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.952181 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.952184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.952512 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.952513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.952558 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.952997 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:24.953002 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:24.954809 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":111,"threads_started":1}
W20260923 22:08:24.955046 11178 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:24.950352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:55475 (local address 127.1.244.254:35397)
0923 22:08:24.950513 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.950515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.950524 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:24.950546 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.950546 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.950556 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.950619 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:24.950967 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.951469 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.952254 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.952447 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.952509 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.954957 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:55475: BlockingRecv error: recv got EOF from 127.1.244.193:55475 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":71,"threads_started":1}
Sep 23 22:08:25 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:25 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:25.094350 11179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:25.089566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:36875)
0923 22:08:25.089713 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:25.089723 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:25.089781 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:25.089926 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:25.089927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:25.090088 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:25.090135 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:25.090137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.090934 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.090935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:25.091468 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:25.091470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.091877 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.091878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:25.091944 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:25.092444 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:25.092449 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:25.094240 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
W20260923 22:08:25.094393 11178 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:25.089633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:36875 (local address 127.1.244.254:35397)
0923 22:08:25.089729 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:25.089731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:25.089768 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:08:25.089838 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:25.089839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:25.089850 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:25.089906 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:25.090238 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.090855 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.091580 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.091788 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.091874 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:25.094326 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:36875: BlockingRecv error: recv got EOF from 127.1.244.194:36875 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 23 22:08:25 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:25 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:25.230947 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:25.226325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:40685)
0923 22:08:25.226481 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:25.226493 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:25.226559 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:25.226702 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:25.226702 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:25.226864 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:25.226913 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:25.226915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.227527 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.227528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:25.228052 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:25.228053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.228390 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.228390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:25.228429 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:25.228863 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:25.228868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:25.230879 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
W20260923 22:08:25.231055 11178 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:25.226409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:40685 (local address 127.1.244.254:35397)
0923 22:08:25.226474 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:25.226476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:25.226545 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:08:25.226616 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:25.226617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:25.226630 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:25.226681 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:25.226976 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.227465 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.228148 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.228327 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.228377 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:25.230960 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:40685: BlockingRecv error: recv got EOF from 127.1.244.195:40685 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 23 22:08:25 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:25 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:25.959659 11181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:25.955283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:41413)
0923 22:08:25.955420 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:25.955431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:25.955516 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:25.955706 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:25.955707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:25.955874 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:25.955922 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:25.955923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.956553 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.956554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:25.957073 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:25.957075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.957382 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.957383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:25.957419 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:25.957852 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:25.957856 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:25.959538 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":53,"threads_started":1}
W20260923 22:08:25.959764 11182 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:25.955432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:41413 (local address 127.1.244.254:35397)
0923 22:08:25.955581 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:25.955584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:25.955592 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:25.955621 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:25.955621 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:25.955631 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:25.955689 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:25.955985 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.956486 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.957139 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:25.957321 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:25.957369 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:25.959669 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:41413: BlockingRecv error: recv got EOF from 127.1.244.193:41413 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":59,"threads_started":1}
Sep 23 22:08:26 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:26 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:26.099617 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.094813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:53515)
0923 22:08:26.095001 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.095012 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.095085 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.095238 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.095239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.095416 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.095463 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.095465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.096333 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.096334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.096868 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.096869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.097242 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.097243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.097300 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.097743 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.097747 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.099507 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
W20260923 22:08:26.099720 11182 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:26.094920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:53515 (local address 127.1.244.254:35397)
0923 22:08:26.094992 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.094994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.095065 (+ 71us) server_negotiation.cc:407] Connection header received
0923 22:08:26.095143 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.095144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.095157 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.095217 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.095578 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.096247 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.096955 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.097156 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.097240 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.099648 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:53515: BlockingRecv error: recv got EOF from 127.1.244.194:53515 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 22:08:26 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:26 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:26.235642 11184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.231371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:55273)
0923 22:08:26.231483 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.231501 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.231572 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.231702 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.231703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.231855 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.231901 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.231903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.232516 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.232516 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.233024 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.233025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.233353 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.233354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.233392 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.233804 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.233808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.235500 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
W20260923 22:08:26.235699 11182 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:26.231442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:55273 (local address 127.1.244.254:35397)
0923 22:08:26.231517 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.231518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.231574 (+ 56us) server_negotiation.cc:407] Connection header received
0923 22:08:26.231616 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.231617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.231638 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.231692 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.231962 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.232452 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.233118 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.233292 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.233344 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.235653 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:55273: BlockingRecv error: recv got EOF from 127.1.244.195:55273 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 23 22:08:26 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:26 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:26.964876 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.960177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:33999)
0923 22:08:26.960323 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.960333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.960386 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.960536 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.960537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.960696 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.960743 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.960745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.961392 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.961392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.961901 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.961903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.962308 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.962308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.962349 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.962791 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.962796 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.964817 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
W20260923 22:08:26.964996 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:26.960241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:33999 (local address 127.1.244.254:35397)
0923 22:08:26.960377 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.960378 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.960388 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:26.960442 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.960443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.960453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.960519 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.960808 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.961327 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.962002 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.962234 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.962308 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.964907 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:33999: BlockingRecv error: recv got EOF from 127.1.244.193:33999 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
Sep 23 22:08:27 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:27 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:27.105089 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.100147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:40493)
0923 22:08:27.100330 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:27.100342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:27.100434 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:27.100570 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:27.100571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:27.100756 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:27.100802 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.100804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.101508 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.101509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.102037 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.102039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.102401 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.102402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.102457 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.103080 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:27.103085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:27.104983 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
W20260923 22:08:27.105125 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:27.100252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:40493 (local address 127.1.244.254:35397)
0923 22:08:27.100327 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.100329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.100422 (+ 93us) server_negotiation.cc:407] Connection header received
0923 22:08:27.100494 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.100495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.100509 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.100556 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:27.100898 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.101414 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.102133 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.102329 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.102411 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.105061 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:40493: BlockingRecv error: recv got EOF from 127.1.244.194:40493 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 22:08:27 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:27 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:27.240849 11188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.236122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:37729)
0923 22:08:27.236303 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:27.236315 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:27.236409 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:27.236576 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:27.236577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:27.236750 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:27.236796 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.236798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.237480 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.237481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.238013 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.238015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.238394 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.238395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.238466 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.238953 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:27.238961 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:27.240739 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":113,"threads_started":1}
W20260923 22:08:27.240897 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:27.236214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:37729 (local address 127.1.244.254:35397)
0923 22:08:27.236293 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.236295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.236389 (+ 94us) server_negotiation.cc:407] Connection header received
0923 22:08:27.236503 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.236504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.236518 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.236560 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:27.236874 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.237387 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.238103 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.238308 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.238393 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.240826 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:37729: BlockingRecv error: recv got EOF from 127.1.244.195:37729 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 23 22:08:27 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:27 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:27.974119 11189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.965522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:53797)
0923 22:08:27.965735 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:27.965748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:27.965855 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:27.966032 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:27.966033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:27.966197 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:27.966247 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.966249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.966965 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.966966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.967499 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.967500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.968036 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.968036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.968079 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.972164 (+ 4085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:27.972169 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:27.974044 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":85,"threads_started":1}
W20260923 22:08:27.974270 11190 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:27.965598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53797 (local address 127.1.244.254:35397)
0923 22:08:27.965756 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.965759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.965816 (+ 57us) server_negotiation.cc:407] Connection header received
0923 22:08:27.965928 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.965929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.965944 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.966013 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:27.966317 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.966900 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.967628 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.967965 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.968016 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.974171 (+ 6155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:53797: BlockingRecv error: recv got EOF from 127.1.244.193:53797 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":56,"threads_started":1}
Sep 23 22:08:28 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:28 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:28.110571 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:28.105646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:45589)
0923 22:08:28.105831 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:28.105843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:28.105932 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:28.106108 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:28.106110 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:28.106298 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:28.106347 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:28.106348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.106976 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.106977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:28.107621 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:28.107623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.107969 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.107970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:28.108072 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:28.108618 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:28.108624 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:28.110512 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
W20260923 22:08:28.110692 11190 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:28.105768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:45589 (local address 127.1.244.254:35397)
0923 22:08:28.105845 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:28.105848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:28.105913 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:08:28.106009 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:28.106010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:28.106024 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:28.106078 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:28.106419 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.106906 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.107690 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.107905 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.107997 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:28.110610 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:45589: BlockingRecv error: recv got EOF from 127.1.244.194:45589 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 23 22:08:28 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:28 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:28.246255 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:28.241381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:52927)
0923 22:08:28.241552 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:28.241569 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:28.241657 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:28.241856 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:28.241857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:28.242043 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:28.242087 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:28.242088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.242757 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.242758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:28.243281 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:28.243282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.243706 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.243707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:28.243772 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:28.244234 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:28.244239 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:28.246147 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
W20260923 22:08:28.246335 11190 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:28.241560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:52927 (local address 127.1.244.254:35397)
0923 22:08:28.241631 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:28.241633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:28.241643 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:28.241758 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:28.241763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:28.241776 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:28.241851 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:28.242170 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.242665 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.243375 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.243623 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.243704 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:28.246252 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:52927: BlockingRecv error: recv got EOF from 127.1.244.195:52927 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 23 22:08:28 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:28 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:28.979439 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:28.974674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:53235)
0923 22:08:28.974793 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:28.974806 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:28.974871 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:28.975005 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:28.975006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:28.975173 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:28.975220 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:28.975223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.976028 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.976029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:28.976570 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:28.976571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.976927 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.976928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:28.976969 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:28.977579 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:28.977584 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:28.979369 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
W20260923 22:08:28.979614 11194 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:28.974738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53235 (local address 127.1.244.254:35397)
0923 22:08:28.974889 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:28.974891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:28.974900 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:28.974923 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:28.974923 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:28.974935 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:28.975000 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:28.975286 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.975950 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.976647 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:28.976859 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:28.976921 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:28.979495 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:53235: BlockingRecv error: recv got EOF from 127.1.244.193:53235 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
Sep 23 22:08:29 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:29 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:29.116245 11195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:29.111083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:56245)
0923 22:08:29.111226 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:29.111236 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:29.111291 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:29.111429 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:29.111429 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:29.111660 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:29.111714 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:29.111716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.112757 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.112758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:29.113516 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:29.113518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.113862 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.113863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:29.113900 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:29.114342 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:29.114346 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:29.116171 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
W20260923 22:08:29.116357 11194 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:29.111167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:56245 (local address 127.1.244.254:35397)
0923 22:08:29.111222 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:29.111224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:29.111279 (+ 55us) server_negotiation.cc:407] Connection header received
0923 22:08:29.111353 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:29.111353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:29.111364 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:29.111408 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:29.111780 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.112646 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.113611 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.113792 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.113852 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:29.116262 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:56245: BlockingRecv error: recv got EOF from 127.1.244.194:56245 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 23 22:08:29 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:29 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:29.251678 11196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:29.246914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:40737)
0923 22:08:29.247120 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:29.247132 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:29.247206 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:29.247373 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:29.247374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:29.247591 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:29.247635 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:29.247637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.248316 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.248317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:29.248873 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:29.248874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.249230 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.249231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:29.249288 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:29.249748 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:29.249753 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:29.251551 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
W20260923 22:08:29.251725 11194 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:29.247019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:40737 (local address 127.1.244.254:35397)
0923 22:08:29.247101 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:29.247103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:29.247187 (+ 84us) server_negotiation.cc:407] Connection header received
0923 22:08:29.247265 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:29.247267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:29.247281 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:29.247351 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:29.247716 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.248214 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.248963 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.249155 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.249239 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:29.251658 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:40737: BlockingRecv error: recv got EOF from 127.1.244.195:40737 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 23 22:08:29 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:29 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:29.984839 11197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:29.980042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:60127)
0923 22:08:29.980341 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:29.980352 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:29.980398 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:29.980544 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:29.980544 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:29.980719 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:29.980767 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:29.980769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.981402 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.981402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:29.981940 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:29.981941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.982266 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.982266 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:29.982308 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:29.982780 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:29.982785 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:29.984727 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":231,"threads_started":1}
W20260923 22:08:29.984886 11198 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:29.980244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:60127 (local address 127.1.244.254:35397)
0923 22:08:29.980375 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:29.980376 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:29.980402 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:08:29.980451 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:29.980451 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:29.980463 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:29.980512 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:29.980836 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.981334 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.982012 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:29.982202 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:29.982251 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:29.984812 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:60127: BlockingRecv error: recv got EOF from 127.1.244.193:60127 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
Sep 23 22:08:30 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:30.121238 11199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.116665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:54267)
0923 22:08:30.116914 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.116933 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.117003 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.117137 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.117137 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.117311 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.117358 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.117360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.117982 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.117983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.118549 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.118551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.118897 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.118898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.118950 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.119402 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.119406 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.121180 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":67,"threads_started":1}
W20260923 22:08:30.121349 11198 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:30.116872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:54267 (local address 127.1.244.254:35397)
0923 22:08:30.116943 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.116944 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.116985 (+ 41us) server_negotiation.cc:407] Connection header received
0923 22:08:30.117063 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.117064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.117076 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.117116 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.117423 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.117908 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.118626 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.118804 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.118854 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.121274 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:54267: BlockingRecv error: recv got EOF from 127.1.244.194:54267 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 23 22:08:30 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:30.256553 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.252148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:45021)
0923 22:08:30.252292 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.252322 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.252379 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.252504 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.252505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.252683 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.252728 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.252730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.253364 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.253365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.253898 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.253900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.254205 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.254206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.254244 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.254683 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.254687 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.256456 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
W20260923 22:08:30.256587 11198 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:30.252316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:45021 (local address 127.1.244.254:35397)
0923 22:08:30.252361 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.252363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.252378 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:08:30.252434 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.252435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.252448 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.252487 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.252791 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.253277 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.253973 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.254146 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.254192 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.256532 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:45021: BlockingRecv error: recv got EOF from 127.1.244.195:45021 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 23 22:08:30 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:30.990134 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.985442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:36601)
0923 22:08:30.985592 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.985610 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.985675 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.985828 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.985828 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.985994 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.986039 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.986041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.986708 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.986708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.987214 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.987215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.987624 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.987624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.987664 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.988105 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.988109 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.990074 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
W20260923 22:08:30.990382 11202 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:30.985519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:36601 (local address 127.1.244.254:35397)
0923 22:08:30.985623 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.985625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.985665 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:08:30.985733 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.985734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.985746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.985810 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.986110 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.986642 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.987286 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.987505 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.987579 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.990180 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:36601: BlockingRecv error: recv got EOF from 127.1.244.193:36601 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
Sep 23 22:08:31 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:31 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:31.126269 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:31.121792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:49535)
0923 22:08:31.121933 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:31.121949 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:31.122000 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:31.122109 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:31.122109 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:31.122273 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:31.122319 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.122321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.122939 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.122940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.123468 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.123469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.123875 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.123876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.123940 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.124346 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:31.124350 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:31.126159 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":56,"threads_started":1}
W20260923 22:08:31.126300 11202 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:31.121930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:49535 (local address 127.1.244.254:35397)
0923 22:08:31.121999 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:31.122001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:31.122010 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:31.122042 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:31.122043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:31.122054 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:31.122097 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:31.122381 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.122876 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.123573 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.123766 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.123847 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.126236 (+ 2389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:49535: BlockingRecv error: recv got EOF from 127.1.244.194:49535 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 23 22:08:31 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:31 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:31.261463 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:31.257038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:35721)
0923 22:08:31.257179 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:31.257194 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:31.257251 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:31.257375 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:31.257376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:31.257551 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:31.257597 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.257599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.258410 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.258411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.258944 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.258946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.259260 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.259261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.259302 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.259746 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:31.259751 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:31.261396 (+ 1645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
W20260923 22:08:31.261554 11202 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:31.257100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:35721 (local address 127.1.244.254:35397)
0923 22:08:31.257160 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:31.257162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:31.257252 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:08:31.257295 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:31.257296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:31.257308 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:31.257358 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:31.257659 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.258340 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.259015 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.259193 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.259242 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.261486 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:35721: BlockingRecv error: recv got EOF from 127.1.244.195:35721 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 23 22:08:31 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:31 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:31.995496 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:31.990782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:32787)
0923 22:08:31.990931 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:31.990946 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:31.991034 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:31.991168 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:31.991169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:31.991351 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:31.991395 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.991397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.992146 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.992147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.992673 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.992675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.993057 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.993058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.993126 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.993647 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:31.993652 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:31.995427 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
W20260923 22:08:31.995690 11206 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:31.990811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:32787 (local address 127.1.244.254:35397)
0923 22:08:31.990968 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:31.990970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:31.991031 (+ 61us) server_negotiation.cc:407] Connection header received
0923 22:08:31.991090 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:31.991091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:31.991104 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:31.991164 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:31.991494 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.992060 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.992773 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.992983 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.993058 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.995524 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:32787: BlockingRecv error: recv got EOF from 127.1.244.193:32787 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
Sep 23 22:08:32 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:32 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:32.131474 11207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:32.126725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:56451)
0923 22:08:32.126888 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:32.126899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:32.126997 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:32.127146 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:32.127147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:32.127326 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:32.127371 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.127373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.128075 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.128077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.128617 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.128619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.128993 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.128995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.129055 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.129483 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:32.129488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:32.131414 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
W20260923 22:08:32.131623 11206 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:32.126800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:56451 (local address 127.1.244.254:35397)
0923 22:08:32.126873 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:32.126876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:32.126979 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:08:32.127054 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:32.127055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:32.127073 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:32.127126 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:32.127462 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.127999 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.128713 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.128911 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.128993 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.131517 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:56451: BlockingRecv error: recv got EOF from 127.1.244.194:56451 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 23 22:08:32 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:32 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:32.266335 11208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:32.261959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.195:45795)
0923 22:08:32.262098 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:32.262118 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:32.262184 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:32.262312 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:32.262313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:32.262484 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:32.262531 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.262533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.263151 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.263151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.263707 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.263709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.264013 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.264014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.264051 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.264474 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:32.264478 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:32.266279 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":53,"threads_started":1}
W20260923 22:08:32.266445 11206 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:32.262050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:45795 (local address 127.1.244.254:35397)
0923 22:08:32.262124 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:32.262125 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:32.262180 (+ 55us) server_negotiation.cc:407] Connection header received
0923 22:08:32.262228 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:32.262228 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:32.262241 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:32.262287 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:32.262593 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.263086 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.263769 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.263944 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.264001 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.266370 (+ 2369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:45795: BlockingRecv error: recv got EOF from 127.1.244.195:45795 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 23 22:08:33 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:33 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:33.005509 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:32.996052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.193:41781)
0923 22:08:32.996167 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:32.996180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:32.996241 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:32.996414 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:32.996415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:32.996626 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:32.996684 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.996687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.997799 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.997799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.998689 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.998691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.999261 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.999262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.999321 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.003091 (+ 3770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:33.003097 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:33.005411 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":65,"threads_started":1}
W20260923 22:08:33.005563 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:32.996116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:41781 (local address 127.1.244.254:35397)
0923 22:08:32.996241 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:32.996243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:32.996255 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:08:32.996309 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:32.996309 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:32.996323 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:32.996403 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:32.996770 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.997703 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.998795 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.999166 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.999245 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.005492 (+ 6247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:41781: BlockingRecv error: recv got EOF from 127.1.244.193:41781 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
Sep 23 22:08:33 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:33 dist-test-slave-b5jz krb5kdc[10151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:33.136307 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.132014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35397 (local address 127.1.244.194:33747)
0923 22:08:33.132178 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:33.132193 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:33.132248 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:33.132360 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:33.132361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:33.132514 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:33.132561 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.132563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.133205 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.133206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.133740 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.133743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.134054 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.134055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.134103 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.134547 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:33.134552 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:33.136241 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35397: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
W20260923 22:08:33.136402 11210 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:33.132060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:33747 (local address 127.1.244.254:35397)
0923 22:08:33.132127 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.132129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.132247 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:08:33.132293 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.132294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.132305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.132354 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.132614 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.133105 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.133811 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.133992 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.134041 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.136331 (+ 2290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:33747: BlockingRecv error: recv got EOF from 127.1.244.194:33747 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:08:33.195137 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 10735
I20260923 22:08:33.199591 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 10886
I20260923 22:08:33.203919 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 11019
I20260923 22:08:33.207945 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 10664
2026-09-23T22:08:33Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17466 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-23T22:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:33Z Disabled control of system clock
I20260923 22:08:33.235251 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37869
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:37869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 22:08:33.315586 11215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:33.315773 11215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:33.315793 11215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:33.316957 11215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:33.316996 11215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:33.317009 11215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:33.317022 11215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:33.317076 11215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:33.318377 11215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:37869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:33.318611 11215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:33.318915 11215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:33.323493 11220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:33.323482 11223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:33.323586 11221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:33.323673 11215 server_base.cc:1034] running on GCE node
I20260923 22:08:33.324041 11215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:33.324332 11215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:33.325522 11215 hybrid_clock.cc:648] HybridClock initialized: now 1790201313325497 us; error 39 us; skew 500 ppm
I20260923 22:08:33.326779 11215 webserver.cc:460] Webserver started at http://127.1.244.254:46521/ using document root <none> and password file <none>
I20260923 22:08:33.327036 11215 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:33.327107 11215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:33.327260 11215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:33.328348 11215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "ca243b2775684994bb17c926d69c37f1"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "9ce9887bccb6c7bab41a131be526a0e0"
server_key_iv: "3d250316976c8a51533667e0c462c92a"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.328712 11215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "ca243b2775684994bb17c926d69c37f1"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "9ce9887bccb6c7bab41a131be526a0e0"
server_key_iv: "3d250316976c8a51533667e0c462c92a"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.329962 11215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:33.330637 11229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.330797 11215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:33.330896 11215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "ca243b2775684994bb17c926d69c37f1"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "9ce9887bccb6c7bab41a131be526a0e0"
server_key_iv: "3d250316976c8a51533667e0c462c92a"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.330977 11215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:33.351027 11215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:33.351364 11215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:33.351521 11215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:33.355499 11215 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:37869
I20260923 22:08:33.355551 11281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:37869 every 8 connection(s)
I20260923 22:08:33.355962 11215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:33.356616 11282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:33.359133 11282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1: Bootstrap starting.
I20260923 22:08:33.359395 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 11215
I20260923 22:08:33.359485 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:33.359735 2003 external_mini_cluster.cc:1428] Setting key b6c3a251e69ced909e303931cf0c8aca
I20260923 22:08:33.359946 11282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:33.360281 11282 log.cc:824] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:33.361006 11282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1: No bootstrap required, opened a new log
I20260923 22:08:33.362891 11282 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca243b2775684994bb17c926d69c37f1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37869 } }
I20260923 22:08:33.363073 11282 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:33.363119 11282 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca243b2775684994bb17c926d69c37f1, State: Initialized, Role: FOLLOWER
I20260923 22:08:33.363243 11282 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca243b2775684994bb17c926d69c37f1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37869 } }
I20260923 22:08:33.363323 11282 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:33.363346 11282 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:33.363404 11282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:33.364421 11282 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca243b2775684994bb17c926d69c37f1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37869 } }
I20260923 22:08:33.364528 11282 leader_election.cc:302] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca243b2775684994bb17c926d69c37f1; no voters:
I20260923 22:08:33.364684 11282 leader_election.cc:288] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:33.364758 11287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:33.364979 11287 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [term 1 LEADER]: Becoming Leader. State: Replica: ca243b2775684994bb17c926d69c37f1, State: Running, Role: LEADER
I20260923 22:08:33.365063 11282 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:33.365571 11287 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca243b2775684994bb17c926d69c37f1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37869 } }
I20260923 22:08:33.366127 11289 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca243b2775684994bb17c926d69c37f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca243b2775684994bb17c926d69c37f1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37869 } } }
I20260923 22:08:33.366272 11289 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:33.366298 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.360277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.1.244.254:37869)
0923 22:08:33.360518 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.360526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.360561 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:08:33.361123 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.361133 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.361208 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.361354 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.361563 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.362941 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.364017 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.364657 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.364789 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.365445 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:33.365449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:33.365449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:33.365688 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:33.365735 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:33.365750 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:33.365817 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:33.365913 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:08:33.366013 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
I20260923 22:08:33.366619 11296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:33.366976 11288 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca243b2775684994bb17c926d69c37f1. Latest consensus state: current_term: 1 leader_uuid: "ca243b2775684994bb17c926d69c37f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca243b2775684994bb17c926d69c37f1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37869 } } }
I20260923 22:08:33.367053 11288 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:33.367205 11296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:33.371157 11296 catalog_manager.cc:1347] Generated new cluster ID: 15598899457845de8ae8d9482e922f8b
I20260923 22:08:33.371213 11296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:33.388803 11296 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:33.389326 11296 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:33.398530 11296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca243b2775684994bb17c926d69c37f1: Generated new TSK 0
I20260923 22:08:33.398695 11296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:33.405988 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37869
--builtin_ntp_servers=127.1.244.212:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 22:08:33.486747 11306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:33.486976 11306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:33.486999 11306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:33.488101 11306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:33.488135 11306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:33.488159 11306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:33.489409 11306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:33.489586 11306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:33.489828 11306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:33.494659 11311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:33.494673 11312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:33.494733 11306 server_base.cc:1034] running on GCE node
W20260923 22:08:33.494673 11314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:33.495090 11306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:33.495360 11306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:33.496522 11306 hybrid_clock.cc:648] HybridClock initialized: now 1790201313496496 us; error 58 us; skew 500 ppm
I20260923 22:08:33.497742 11306 webserver.cc:460] Webserver started at http://127.1.244.193:38419/ using document root <none> and password file <none>
I20260923 22:08:33.497998 11306 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:33.498067 11306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:33.498203 11306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:33.499239 11306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "5bb2b9c582324726a5d180aea9f5ac7c"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "1431bbe2dfc0b2077bd4dfdea9bdaab3"
server_key_iv: "a2abf32fb909641a3e8c56654f151724"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.499671 11306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "5bb2b9c582324726a5d180aea9f5ac7c"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "1431bbe2dfc0b2077bd4dfdea9bdaab3"
server_key_iv: "a2abf32fb909641a3e8c56654f151724"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.501005 11306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:33.501694 11320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.501868 11306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:33.501963 11306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "5bb2b9c582324726a5d180aea9f5ac7c"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "1431bbe2dfc0b2077bd4dfdea9bdaab3"
server_key_iv: "a2abf32fb909641a3e8c56654f151724"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.502058 11306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:33.517477 11306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:33.517748 11306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:33.517908 11306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:33.518114 11306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:33.518424 11306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:33.518477 11306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.518510 11306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:33.518536 11306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.524255 11306 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:41427
I20260923 22:08:33.524290 11433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:41427 every 8 connection(s)
I20260923 22:08:33.524688 11306 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:33.529518 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.525394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:60481 (local address 127.1.244.254:37869)
0923 22:08:33.525474 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.525476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.525684 (+ 208us) server_negotiation.cc:407] Connection header received
0923 22:08:33.526126 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.526127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.526137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.526187 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.526795 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.527345 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.528314 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.528495 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.528548 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.529332 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:33.529332 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:33.529333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:33.529336 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:33.529346 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:33.529349 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:33.529386 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:33.529445 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 22:08:33.529461 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:08:33.529770 11436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.525361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37869 (local address 127.1.244.193:60481)
0923 22:08:33.525569 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:33.525602 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:33.526046 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:33.526240 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:33.526244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:33.526252 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:33.526730 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.526734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.527421 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.527422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.528236 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.528239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.528571 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.528572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.528672 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.529210 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:33.529210 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:33.529237 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:33.529258 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:33.529388 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:33.529390 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:33.529392 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:33.529438 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:08:33.529557 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260923 22:08:33.530231 11434 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37869
I20260923 22:08:33.530316 11434 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:33.530454 11434 heartbeater.cc:511] Master 127.1.244.254:37869 requested a full tablet report, sending...
I20260923 22:08:33.530646 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 11306
I20260923 22:08:33.530741 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:33.530952 2003 external_mini_cluster.cc:1428] Setting key 3e1b91c8f5ea982d51fef5f483978099
I20260923 22:08:33.531014 11246 ts_manager.cc:194] Registered new tserver with Master: 5bb2b9c582324726a5d180aea9f5ac7c (127.1.244.193:41427)
I20260923 22:08:33.531713 11246 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.193:60481
I20260923 22:08:33.532142 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37869
--builtin_ntp_servers=127.1.244.212:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 22:08:33.611637 11437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:33.611835 11437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:33.611857 11437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:33.613018 11437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:33.613059 11437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:33.613109 11437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:33.614429 11437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:33.614668 11437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:33.614961 11437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:33.619676 11443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:33.619735 11437 server_base.cc:1034] running on GCE node
W20260923 22:08:33.619668 11442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:33.619654 11445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:33.620079 11437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:33.620267 11437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:33.621407 11437 hybrid_clock.cc:648] HybridClock initialized: now 1790201313621385 us; error 41 us; skew 500 ppm
I20260923 22:08:33.622507 11437 webserver.cc:460] Webserver started at http://127.1.244.194:43169/ using document root <none> and password file <none>
I20260923 22:08:33.622687 11437 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:33.622726 11437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:33.622820 11437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:33.623857 11437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "11c7ea58d77d4a2ead7d1f096cc440c8"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "cb1302c74c0637c85e80c87bdfaa8d39"
server_key_iv: "1dbc295c1d426f72990e3eb2ae17753d"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.624145 11437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "11c7ea58d77d4a2ead7d1f096cc440c8"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "cb1302c74c0637c85e80c87bdfaa8d39"
server_key_iv: "1dbc295c1d426f72990e3eb2ae17753d"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.625337 11437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:33.625962 11451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.626121 11437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:33.626219 11437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "11c7ea58d77d4a2ead7d1f096cc440c8"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "cb1302c74c0637c85e80c87bdfaa8d39"
server_key_iv: "1dbc295c1d426f72990e3eb2ae17753d"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.626299 11437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:33.633306 11437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:33.633558 11437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:33.633698 11437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:33.633940 11437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:33.634243 11437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:33.634294 11437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.634326 11437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:33.634352 11437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.639979 11437 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:37485
I20260923 22:08:33.640017 11564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:37485 every 8 connection(s)
I20260923 22:08:33.640331 11437 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:33.645059 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.641044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:39845 (local address 127.1.244.254:37869)
0923 22:08:33.641103 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.641105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.641317 (+ 212us) server_negotiation.cc:407] Connection header received
0923 22:08:33.641750 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.641751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.641761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.641815 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.642429 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.642909 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.643929 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.644109 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.644163 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.644850 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:33.644852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:33.644852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:33.644856 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:33.644868 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:33.644872 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:33.644914 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:33.644984 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:08:33.645003 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:08:33.645501 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.641031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37869 (local address 127.1.244.194:39845)
0923 22:08:33.641198 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:33.641235 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:33.641676 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:33.641865 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:33.641868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:33.641876 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:33.642362 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.642366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.642986 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.642986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.643857 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.643860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.644181 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.644182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.644285 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.644717 (+ 432us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:33.644718 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:33.644744 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:33.644765 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:33.644914 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:33.644917 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:33.644919 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:33.644964 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:08:33.645036 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260923 22:08:33.646018 11565 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37869
I20260923 22:08:33.646135 11565 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:33.646340 11565 heartbeater.cc:511] Master 127.1.244.254:37869 requested a full tablet report, sending...
I20260923 22:08:33.646615 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 11437
I20260923 22:08:33.646727 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:33.646783 11246 ts_manager.cc:194] Registered new tserver with Master: 11c7ea58d77d4a2ead7d1f096cc440c8 (127.1.244.194:37485)
I20260923 22:08:33.646952 2003 external_mini_cluster.cc:1428] Setting key e13928ed662c1de274aae251f580a713
I20260923 22:08:33.647236 11246 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.194:39845
I20260923 22:08:33.648043 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37869
--builtin_ntp_servers=127.1.244.212:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 22:08:33.728971 11568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:33.729174 11568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:33.729197 11568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:33.730535 11568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:33.730577 11568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:33.730604 11568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:33.732075 11568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:33.732327 11568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:33.732627 11568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:33.737761 11576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:33.737933 11568 server_base.cc:1034] running on GCE node
W20260923 22:08:33.737762 11573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:33.737787 11574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:33.738305 11568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:33.738586 11568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:33.739758 11568 hybrid_clock.cc:648] HybridClock initialized: now 1790201313739735 us; error 35 us; skew 500 ppm
I20260923 22:08:33.740963 11568 webserver.cc:460] Webserver started at http://127.1.244.195:37419/ using document root <none> and password file <none>
I20260923 22:08:33.741225 11568 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:33.741297 11568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:33.741449 11568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:33.742512 11568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "00e9fd3511ac4f89b14af01a848e365b"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "fe1a629e938766781e08c00868ae4e4f"
server_key_iv: "d5aeee30d40691dc4aab6d94114b3bad"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.742913 11568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "00e9fd3511ac4f89b14af01a848e365b"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "fe1a629e938766781e08c00868ae4e4f"
server_key_iv: "d5aeee30d40691dc4aab6d94114b3bad"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.744279 11568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:33.745028 11582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.745214 11568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:33.745327 11568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "00e9fd3511ac4f89b14af01a848e365b"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "fe1a629e938766781e08c00868ae4e4f"
server_key_iv: "d5aeee30d40691dc4aab6d94114b3bad"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.745406 11568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:33.757664 11568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:33.758059 11568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:33.758224 11568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:33.758450 11568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:33.758765 11568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:33.758824 11568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.758868 11568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:33.758899 11568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.765183 11568 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:36021
I20260923 22:08:33.765223 11695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:36021 every 8 connection(s)
I20260923 22:08:33.765643 11568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:33.771138 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.766562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:38153 (local address 127.1.244.254:37869)
0923 22:08:33.766614 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.766617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.766888 (+ 271us) server_negotiation.cc:407] Connection header received
0923 22:08:33.767583 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.767585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.767599 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.767673 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.768364 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.768854 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.769821 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.770035 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.770107 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.770863 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:33.770864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:33.770864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:33.770869 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:33.770883 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:33.770887 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:33.770951 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:33.771036 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:08:33.771061 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:33.771356 11698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.766566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37869 (local address 127.1.244.195:38153)
0923 22:08:33.766765 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:33.766796 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:33.767345 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:33.767782 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:33.767796 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:33.767804 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:33.768262 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.768266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.768947 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.768948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.769709 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.769711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.770125 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.770126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.770245 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.770716 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:33.770717 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:33.770742 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:33.770762 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:33.770953 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:33.770958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:33.770961 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:33.771039 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:08:33.771133 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260923 22:08:33.771827 11696 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37869
I20260923 22:08:33.771924 11696 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:33.772090 11696 heartbeater.cc:511] Master 127.1.244.254:37869 requested a full tablet report, sending...
I20260923 22:08:33.772496 11245 ts_manager.cc:194] Registered new tserver with Master: 00e9fd3511ac4f89b14af01a848e365b (127.1.244.195:36021)
I20260923 22:08:33.772787 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 11568
I20260923 22:08:33.772887 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:33.772928 11245 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.195:38153
I20260923 22:08:33.773101 2003 external_mini_cluster.cc:1428] Setting key d43048b4b9ad4c523422ea2242846465
I20260923 22:08:33.774034 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:08:33.775756 11285 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:33.775360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.1.244.254:37869)
0923 22:08:33.775408 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.775410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.775489 (+ 79us) server_negotiation.cc:407] Connection header received
0923 22:08:33.775553 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.775554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.775561 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.775595 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.775716 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35146: BlockingRecv error: recv got EOF from 127.0.0.1:35146 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:08:33.775828 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 11306
I20260923 22:08:33.781478 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 11437
I20260923 22:08:33.785840 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 11568
I20260923 22:08:33.789851 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 11215
2026-09-23T22:08:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (580 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-23T22:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:33Z Disabled control of system clock
I20260923 22:08:33.815341 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33981
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:33981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:08:33.895284 11707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:33.895840 11707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:33.895897 11707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:33.898183 11707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:33.898245 11707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:33.898264 11707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:33.898288 11707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:33.898312 11707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:33.901063 11707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:33981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:33.901546 11707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:33.901906 11707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:33.908389 11715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:33.908318 11713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:33.908736 11707 server_base.cc:1034] running on GCE node
W20260923 22:08:33.908779 11712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:33.909140 11707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:33.909485 11707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:33.910653 11707 hybrid_clock.cc:648] HybridClock initialized: now 1790201313910636 us; error 33 us; skew 500 ppm
I20260923 22:08:33.912233 11707 webserver.cc:460] Webserver started at http://127.1.244.254:35971/ using document root <none> and password file <none>
I20260923 22:08:33.912549 11707 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:33.912621 11707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:33.912766 11707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:33.914283 11707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "ec50abfcb5c244828e85c4af40476fcf"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "86f8945804a17efc90876fa8ab7b9237"
server_key_iv: "a01cc7eafa00dbc165d670c90bb28018"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.914749 11707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "ec50abfcb5c244828e85c4af40476fcf"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "86f8945804a17efc90876fa8ab7b9237"
server_key_iv: "a01cc7eafa00dbc165d670c90bb28018"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.916533 11707 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260923 22:08:33.917490 11721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:33.917675 11707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:33.917770 11707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "ec50abfcb5c244828e85c4af40476fcf"
format_stamp: "Formatted at 2026-09-23 22:08:33 on dist-test-slave-b5jz"
server_key: "86f8945804a17efc90876fa8ab7b9237"
server_key_iv: "a01cc7eafa00dbc165d670c90bb28018"
server_key_version: "encryptionkey@0"
I20260923 22:08:33.917856 11707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:33.921497 11707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:33.921747 11707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:33.921883 11707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:33.925521 11707 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:33981
I20260923 22:08:33.925565 11773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:33981 every 8 connection(s)
I20260923 22:08:33.925864 11707 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:33.926491 11774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:33.929111 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 11707
I20260923 22:08:33.929216 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:33.929196 11774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf: Bootstrap starting.
I20260923 22:08:33.929416 2003 external_mini_cluster.cc:1428] Setting key acd2be722e8b54d6baad45828151b81d
I20260923 22:08:33.929904 11774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:33.930231 11774 log.cc:824] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:33.930933 11774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf: No bootstrap required, opened a new log
I20260923 22:08:33.932000 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.929908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.1.244.254:33981)
0923 22:08:33.930182 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.930191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.930222 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:08:33.930743 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.930751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.930797 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.930933 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.931584 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:33.931587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:33.931588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:33.931592 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:33.931634 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:33.931649 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:33.931740 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:33.931770 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 22:08:33.931824 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":82,"threads_started":1}
I20260923 22:08:33.932723 11774 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec50abfcb5c244828e85c4af40476fcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33981 } }
I20260923 22:08:33.932842 11774 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:33.932862 11774 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec50abfcb5c244828e85c4af40476fcf, State: Initialized, Role: FOLLOWER
I20260923 22:08:33.932968 11774 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec50abfcb5c244828e85c4af40476fcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33981 } }
I20260923 22:08:33.933044 11774 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:33.933065 11774 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:33.933091 11774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:33.933825 11774 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec50abfcb5c244828e85c4af40476fcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33981 } }
I20260923 22:08:33.933947 11774 leader_election.cc:302] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec50abfcb5c244828e85c4af40476fcf; no voters:
I20260923 22:08:33.934118 11774 leader_election.cc:288] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:33.934150 11779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:33.934247 11779 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [term 1 LEADER]: Becoming Leader. State: Replica: ec50abfcb5c244828e85c4af40476fcf, State: Running, Role: LEADER
I20260923 22:08:33.934327 11779 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec50abfcb5c244828e85c4af40476fcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33981 } }
I20260923 22:08:33.934373 11774 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:33.934633 11779 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec50abfcb5c244828e85c4af40476fcf. Latest consensus state: current_term: 1 leader_uuid: "ec50abfcb5c244828e85c4af40476fcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec50abfcb5c244828e85c4af40476fcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33981 } } }
I20260923 22:08:33.934707 11779 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:33.934643 11780 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec50abfcb5c244828e85c4af40476fcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec50abfcb5c244828e85c4af40476fcf" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33981 } } }
I20260923 22:08:33.934749 11780 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:33.935688 11784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:33.936144 11784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:33.937515 11784 catalog_manager.cc:1347] Generated new cluster ID: dbb4bbbfae914f27ad787d5241d73413
I20260923 22:08:33.937572 11784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:33.951984 11784 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:33.952153 11784 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:33.959201 11784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec50abfcb5c244828e85c4af40476fcf: Generated new TSK 0
I20260923 22:08:33.959380 11784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:33.967211 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33981
--builtin_ntp_servers=127.1.244.212:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:08:34.057788 11798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:34.058008 11798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:34.058045 11798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:34.059191 11798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:34.059254 11798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:34.059297 11798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:34.060683 11798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:34.060940 11798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:34.061228 11798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:34.067106 11804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:34.067304 11798 server_base.cc:1034] running on GCE node
W20260923 22:08:34.067106 11806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:34.067098 11803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:34.067642 11798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:34.067935 11798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:34.069139 11798 hybrid_clock.cc:648] HybridClock initialized: now 1790201314069097 us; error 56 us; skew 500 ppm
I20260923 22:08:34.070514 11798 webserver.cc:460] Webserver started at http://127.1.244.193:37439/ using document root <none> and password file <none>
I20260923 22:08:34.070797 11798 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:34.070868 11798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:34.071014 11798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:34.072149 11798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "72ba8e2fa21b4f088c06153f870fe614"
format_stamp: "Formatted at 2026-09-23 22:08:34 on dist-test-slave-b5jz"
server_key: "56ff84ae1a965d0bd41873f3cb52ac7f"
server_key_iv: "128a87b4163c64b308d23dac02cb9957"
server_key_version: "encryptionkey@0"
I20260923 22:08:34.072513 11798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "72ba8e2fa21b4f088c06153f870fe614"
format_stamp: "Formatted at 2026-09-23 22:08:34 on dist-test-slave-b5jz"
server_key: "56ff84ae1a965d0bd41873f3cb52ac7f"
server_key_iv: "128a87b4163c64b308d23dac02cb9957"
server_key_version: "encryptionkey@0"
I20260923 22:08:34.073814 11798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:34.074609 11812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:34.074795 11798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:34.074892 11798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "72ba8e2fa21b4f088c06153f870fe614"
format_stamp: "Formatted at 2026-09-23 22:08:34 on dist-test-slave-b5jz"
server_key: "56ff84ae1a965d0bd41873f3cb52ac7f"
server_key_iv: "128a87b4163c64b308d23dac02cb9957"
server_key_version: "encryptionkey@0"
I20260923 22:08:34.074970 11798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:34.078435 11798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:34.078703 11798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:34.078840 11798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:34.079041 11798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:34.079339 11798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:34.079387 11798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:34.079427 11798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:34.079453 11798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:34.085140 11798 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:36139
I20260923 22:08:34.085181 11925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:36139 every 8 connection(s)
I20260923 22:08:34.085467 11798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:34.088296 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:34.086156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:35747 (local address 127.1.244.254:33981)
0923 22:08:34.086211 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:34.086213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:34.086427 (+ 214us) server_negotiation.cc:407] Connection header received
0923 22:08:34.086930 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:34.086931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:34.086937 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:34.086997 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:34.088117 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:34.088119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:34.088119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:34.088123 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:34.088137 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:34.088141 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:34.088186 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:34.088233 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:08:34.088245 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:08:34.088480 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:34.086127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33981 (local address 127.1.244.193:35747)
0923 22:08:34.086302 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:34.086345 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:34.086788 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:34.087075 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:34.087080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:34.087089 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:34.087984 (+ 895us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:34.087984 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:34.088010 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:34.088037 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:34.088185 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:34.088188 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:34.088190 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:34.088240 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:08:34.088284 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260923 22:08:34.088937 11926 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33981
I20260923 22:08:34.089035 11926 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:34.089175 11926 heartbeater.cc:511] Master 127.1.244.254:33981 requested a full tablet report, sending...
I20260923 22:08:34.089674 11738 ts_manager.cc:194] Registered new tserver with Master: 72ba8e2fa21b4f088c06153f870fe614 (127.1.244.193:36139)
I20260923 22:08:34.092537 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 11798
I20260923 22:08:34.092613 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:34.092759 2003 external_mini_cluster.cc:1428] Setting key 7cd5ae8430bc7721fe3259d9e1788655
I20260923 22:08:34.093590 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33981
--builtin_ntp_servers=127.1.244.212:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:08:34.174388 11929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:34.174585 11929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:34.174607 11929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:34.175794 11929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:34.175834 11929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:34.175859 11929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:34.177253 11929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:34.177486 11929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:34.177778 11929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:34.182626 11934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:34.182662 11937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:34.182662 11935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:34.182670 11929 server_base.cc:1034] running on GCE node
I20260923 22:08:34.182920 11929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:34.183185 11929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:34.184350 11929 hybrid_clock.cc:648] HybridClock initialized: now 1790201314184339 us; error 45 us; skew 500 ppm
I20260923 22:08:34.185582 11929 webserver.cc:460] Webserver started at http://127.1.244.194:44709/ using document root <none> and password file <none>
I20260923 22:08:34.185822 11929 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:34.185894 11929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:34.186049 11929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:34.187063 11929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "ee11e04cbe59460faebe9eb7bfa0b89e"
format_stamp: "Formatted at 2026-09-23 22:08:34 on dist-test-slave-b5jz"
server_key: "18dad6f2082498ea067f0be58ea36c6d"
server_key_iv: "eff80cec1cb0a6de887025129df8eb5a"
server_key_version: "encryptionkey@0"
I20260923 22:08:34.187435 11929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "ee11e04cbe59460faebe9eb7bfa0b89e"
format_stamp: "Formatted at 2026-09-23 22:08:34 on dist-test-slave-b5jz"
server_key: "18dad6f2082498ea067f0be58ea36c6d"
server_key_iv: "eff80cec1cb0a6de887025129df8eb5a"
server_key_version: "encryptionkey@0"
I20260923 22:08:34.188735 11929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:34.189429 11943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:34.189605 11929 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:34.189699 11929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "ee11e04cbe59460faebe9eb7bfa0b89e"
format_stamp: "Formatted at 2026-09-23 22:08:34 on dist-test-slave-b5jz"
server_key: "18dad6f2082498ea067f0be58ea36c6d"
server_key_iv: "eff80cec1cb0a6de887025129df8eb5a"
server_key_version: "encryptionkey@0"
I20260923 22:08:34.189782 11929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:34.192975 11929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:34.193225 11929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:34.193375 11929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:34.193578 11929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:34.193866 11929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:34.193920 11929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:34.193964 11929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:34.193992 11929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:34.199325 11929 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:46799
I20260923 22:08:34.199386 12056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:46799 every 8 connection(s)
I20260923 22:08:34.199723 11929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:34.202339 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:34.200382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:45633 (local address 127.1.244.254:33981)
0923 22:08:34.200435 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:34.200438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:34.200679 (+ 241us) server_negotiation.cc:407] Connection header received
0923 22:08:34.201072 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:34.201073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:34.201078 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:34.201126 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:34.202163 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:34.202164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:34.202164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:34.202168 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:34.202180 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:34.202184 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:34.202229 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:34.202275 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:08:34.202288 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:08:34.202538 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:34.200379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33981 (local address 127.1.244.194:45633)
0923 22:08:34.200563 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:34.200595 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:34.200995 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:34.201182 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:34.201185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:34.201192 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:34.202002 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:34.202002 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:34.202028 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:34.202055 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:34.202228 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:34.202230 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:34.202232 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:34.202280 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:08:34.202325 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260923 22:08:34.203014 12057 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33981
I20260923 22:08:34.203107 12057 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:34.203253 12057 heartbeater.cc:511] Master 127.1.244.254:33981 requested a full tablet report, sending...
I20260923 22:08:34.203672 11738 ts_manager.cc:194] Registered new tserver with Master: ee11e04cbe59460faebe9eb7bfa0b89e (127.1.244.194:46799)
I20260923 22:08:34.207768 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 11929
I20260923 22:08:34.207867 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:34.208060 2003 external_mini_cluster.cc:1428] Setting key 32f0fcd8220eb2c02c5521cfa4894647
I20260923 22:08:34.208902 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33981
--builtin_ntp_servers=127.1.244.212:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:08:34.288169 12060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:34.288391 12060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:34.288425 12060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:34.289556 12060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:34.289619 12060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:34.289661 12060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:34.290982 12060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:34.291234 12060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:34.291517 12060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:34.296295 12068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:34.296310 12066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:34.296295 12065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:34.296492 12060 server_base.cc:1034] running on GCE node
I20260923 22:08:34.296734 12060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:34.296999 12060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:34.298134 12060 hybrid_clock.cc:648] HybridClock initialized: now 1790201314298125 us; error 29 us; skew 500 ppm
I20260923 22:08:34.299254 12060 webserver.cc:460] Webserver started at http://127.1.244.195:46821/ using document root <none> and password file <none>
I20260923 22:08:34.299468 12060 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:34.299510 12060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:34.299670 12060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:34.300647 12060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "c5cb56241f4844a8bae3a7c282b2ebbc"
format_stamp: "Formatted at 2026-09-23 22:08:34 on dist-test-slave-b5jz"
server_key: "8f5daf9b661291e630277268f2215695"
server_key_iv: "4affee1346023eb93f84debbe2d4f3ec"
server_key_version: "encryptionkey@0"
I20260923 22:08:34.300997 12060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "c5cb56241f4844a8bae3a7c282b2ebbc"
format_stamp: "Formatted at 2026-09-23 22:08:34 on dist-test-slave-b5jz"
server_key: "8f5daf9b661291e630277268f2215695"
server_key_iv: "4affee1346023eb93f84debbe2d4f3ec"
server_key_version: "encryptionkey@0"
I20260923 22:08:34.302220 12060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:34.302959 12074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:34.303149 12060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:34.303239 12060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "c5cb56241f4844a8bae3a7c282b2ebbc"
format_stamp: "Formatted at 2026-09-23 22:08:34 on dist-test-slave-b5jz"
server_key: "8f5daf9b661291e630277268f2215695"
server_key_iv: "4affee1346023eb93f84debbe2d4f3ec"
server_key_version: "encryptionkey@0"
I20260923 22:08:34.303316 12060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:34.306532 12060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:34.306769 12060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:34.306911 12060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:34.307113 12060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:34.307410 12060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:34.307459 12060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:34.307495 12060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:34.307523 12060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:34.313061 12060 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:35607
I20260923 22:08:34.313113 12187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:35607 every 8 connection(s)
I20260923 22:08:34.313392 12060 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:34.316202 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:34.314002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:57901 (local address 127.1.244.254:33981)
0923 22:08:34.314045 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:34.314047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:34.314293 (+ 246us) server_negotiation.cc:407] Connection header received
0923 22:08:34.314747 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:34.314749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:34.314757 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:34.314814 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:34.315944 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:34.315946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:34.315947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:34.315952 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:34.315967 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:34.315971 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:34.316023 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:34.316120 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 22:08:34.316136 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:08:34.316390 12190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:34.313988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33981 (local address 127.1.244.195:57901)
0923 22:08:34.314179 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:34.314208 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:34.314642 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:34.314876 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:34.314879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:34.314888 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:34.315799 (+ 911us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:34.315800 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:34.315825 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:34.315850 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:34.316028 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:34.316033 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:34.316036 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:34.316117 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:08:34.316174 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260923 22:08:34.316910 12188 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33981
I20260923 22:08:34.317026 12188 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:34.317174 12188 heartbeater.cc:511] Master 127.1.244.254:33981 requested a full tablet report, sending...
I20260923 22:08:34.317508 11738 ts_manager.cc:194] Registered new tserver with Master: c5cb56241f4844a8bae3a7c282b2ebbc (127.1.244.195:35607)
I20260923 22:08:34.323297 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 12060
I20260923 22:08:34.323410 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:34.323688 2003 external_mini_cluster.cc:1428] Setting key a57785b14c38bbcc1a0d5842d80b7cbf
I20260923 22:08:34.324505 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:08:34.326210 11777 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:34.325849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48008 (local address 127.1.244.254:33981)
0923 22:08:34.325904 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:34.325906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:34.325988 (+ 82us) server_negotiation.cc:407] Connection header received
0923 22:08:34.326027 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:34.326028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:34.326032 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:34.326079 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:34.326167 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48008: BlockingRecv error: recv got EOF from 127.0.0.1:48008 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:08:34.326346 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 11798
I20260923 22:08:34.330497 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 11929
I20260923 22:08:34.334947 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 12060
I20260923 22:08:34.339073 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 11707
2026-09-23T22:08:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (550 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:34 dist-test-slave-b5jz krb5kdc[12198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:34 dist-test-slave-b5jz krb5kdc[12198](info): set up 2 sockets
Sep 23 22:08:34 dist-test-slave-b5jz krb5kdc[12198](info): commencing operation
krb5kdc: starting...
W20260923 22:08:35.396641 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:08:35 dist-test-slave-b5jz krb5kdc[12198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:08:35.510504 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35633
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:35.590811 12211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:35.591051 12211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:35.591090 12211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:35.592330 12211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:35.592396 12211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:35.592418 12211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:35.592439 12211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:35.592469 12211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:35.593927 12211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:35.594210 12211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:35.594440 12211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:35.599174 12217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:35.599188 12216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:35.599182 12219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:35.599308 12211 server_base.cc:1034] running on GCE node
I20260923 22:08:35.599654 12211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:35.600072 12211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:35.601243 12211 hybrid_clock.cc:648] HybridClock initialized: now 1790201315601217 us; error 43 us; skew 500 ppm
Sep 23 22:08:35 dist-test-slave-b5jz krb5kdc[12198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201315, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:35.603623 12211 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:08:35.604060 12211 webserver.cc:460] Webserver started at http://127.1.244.254:42537/ using document root <none> and password file <none>
I20260923 22:08:35.604301 12211 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:35.604369 12211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:35.604502 12211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:35.605537 12211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "89d9ed4bdfd04d0b921e58f3854169de"
format_stamp: "Formatted at 2026-09-23 22:08:35 on dist-test-slave-b5jz"
server_key: "e4f15312aa7900f121c1f2a7e098b716"
server_key_iv: "d9a283aa33a1df5c5eb198209d3f79c9"
server_key_version: "encryptionkey@0"
I20260923 22:08:35.605844 12211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "89d9ed4bdfd04d0b921e58f3854169de"
format_stamp: "Formatted at 2026-09-23 22:08:35 on dist-test-slave-b5jz"
server_key: "e4f15312aa7900f121c1f2a7e098b716"
server_key_iv: "d9a283aa33a1df5c5eb198209d3f79c9"
server_key_version: "encryptionkey@0"
I20260923 22:08:35.607066 12211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:35.607755 12226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:35.607928 12211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:35.608027 12211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "89d9ed4bdfd04d0b921e58f3854169de"
format_stamp: "Formatted at 2026-09-23 22:08:35 on dist-test-slave-b5jz"
server_key: "e4f15312aa7900f121c1f2a7e098b716"
server_key_iv: "d9a283aa33a1df5c5eb198209d3f79c9"
server_key_version: "encryptionkey@0"
I20260923 22:08:35.608115 12211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:35.617646 12211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:35.619352 12211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:35.619514 12211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:35.623139 12211 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35633
I20260923 22:08:35.623173 12278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35633 every 8 connection(s)
I20260923 22:08:35.623499 12211 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:35.624503 12279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:35.624881 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 12211
I20260923 22:08:35.624986 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:08:35.625213 2003 external_mini_cluster.cc:1428] Setting key cedb793880532adb0bebd88dcab29d3c
I20260923 22:08:35.627394 12279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de: Bootstrap starting.
I20260923 22:08:35.628060 12279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:35.628355 12279 log.cc:824] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:35.629071 12279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de: No bootstrap required, opened a new log
I20260923 22:08:35.631515 12279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89d9ed4bdfd04d0b921e58f3854169de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35633 } }
I20260923 22:08:35.631716 12279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:35.631744 12279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89d9ed4bdfd04d0b921e58f3854169de, State: Initialized, Role: FOLLOWER
I20260923 22:08:35.631886 12279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89d9ed4bdfd04d0b921e58f3854169de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35633 } }
I20260923 22:08:35.631973 12279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:35.631999 12279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:35.632064 12279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [term 0 FOLLOWER]: Advancing to term 1
Sep 23 22:08:35 dist-test-slave-b5jz krb5kdc[12198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:35.632669 12279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89d9ed4bdfd04d0b921e58f3854169de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35633 } }
I20260923 22:08:35.632781 12279 leader_election.cc:302] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89d9ed4bdfd04d0b921e58f3854169de; no voters:
I20260923 22:08:35.632934 12279 leader_election.cc:288] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:35.632997 12284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:35.633173 12284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [term 1 LEADER]: Becoming Leader. State: Replica: 89d9ed4bdfd04d0b921e58f3854169de, State: Running, Role: LEADER
I20260923 22:08:35.633229 12279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:35.633322 12284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89d9ed4bdfd04d0b921e58f3854169de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35633 } }
I20260923 22:08:35.633677 12285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89d9ed4bdfd04d0b921e58f3854169de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89d9ed4bdfd04d0b921e58f3854169de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35633 } } }
I20260923 22:08:35.634187 12285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:35.633682 12284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89d9ed4bdfd04d0b921e58f3854169de. Latest consensus state: current_term: 1 leader_uuid: "89d9ed4bdfd04d0b921e58f3854169de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89d9ed4bdfd04d0b921e58f3854169de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35633 } } }
I20260923 22:08:35.634333 12284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:35.634509 12297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:35.635133 12297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:35.636809 12297 catalog_manager.cc:1347] Generated new cluster ID: bc1bb6cfa20f43d782532746e52f50ac
I20260923 22:08:35.636916 12297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:35.636957 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:35.625762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34744 (local address 127.1.244.254:35633)
0923 22:08:35.625910 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:35.625918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:35.625942 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:08:35.626325 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:35.626331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:35.626374 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:35.626485 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:35.627248 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.628291 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.629316 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.629709 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.629785 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.632600 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:35.632604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:35.632605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:35.632616 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:35.634908 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:35.635346 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:35.635349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:35.635349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:35.635438 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:35.635704 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:35.635704 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:35.635705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:35.636005 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:35.636172 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:35.636630 (+ 458us) server_negotiation.cc:299] Negotiation successful
0923 22:08:35.636764 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260923 22:08:35.657586 12297 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:35.658305 12297 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:35.667507 12297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de: Generated new TSK 0
I20260923 22:08:35.667714 12297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:08:35.695317 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35633
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:35.775192 12305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:35.775420 12305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:35.775457 12305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:35.776618 12305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:35.776682 12305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:35.776727 12305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:08:35.778215 12305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:35.778470 12305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:35.778695 12305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:35.783474 12310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:35.783576 12305 server_base.cc:1034] running on GCE node
W20260923 22:08:35.783492 12311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:35.783512 12313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:35.783916 12305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:35.784160 12305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:35.785331 12305 hybrid_clock.cc:648] HybridClock initialized: now 1790201315785324 us; error 36 us; skew 500 ppm
Sep 23 22:08:35 dist-test-slave-b5jz krb5kdc[12198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201315, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:35.787738 12305 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:08:35.788156 12305 webserver.cc:460] Webserver started at http://127.1.244.193:45643/ using document root <none> and password file <none>
I20260923 22:08:35.788390 12305 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:35.788460 12305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:35.788615 12305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:35.789636 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "c29337207e2c4b3aa39f7340278a8271"
format_stamp: "Formatted at 2026-09-23 22:08:35 on dist-test-slave-b5jz"
server_key: "a4b57dcda686f3753986e590d77d7979"
server_key_iv: "4c498d39eedc253301b72f0efceaa727"
server_key_version: "encryptionkey@0"
I20260923 22:08:35.789994 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "c29337207e2c4b3aa39f7340278a8271"
format_stamp: "Formatted at 2026-09-23 22:08:35 on dist-test-slave-b5jz"
server_key: "a4b57dcda686f3753986e590d77d7979"
server_key_iv: "4c498d39eedc253301b72f0efceaa727"
server_key_version: "encryptionkey@0"
I20260923 22:08:35.791251 12305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:35.792024 12320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:35.792191 12305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:35.792287 12305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "c29337207e2c4b3aa39f7340278a8271"
format_stamp: "Formatted at 2026-09-23 22:08:35 on dist-test-slave-b5jz"
server_key: "a4b57dcda686f3753986e590d77d7979"
server_key_iv: "4c498d39eedc253301b72f0efceaa727"
server_key_version: "encryptionkey@0"
I20260923 22:08:35.792367 12305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:35.800390 12305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:35.802198 12305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:35.802372 12305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:35.802595 12305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:35.802891 12305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:35.802944 12305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:35.802994 12305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:35.803041 12305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:35.808912 12305 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:36577
I20260923 22:08:35.808952 12433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:36577 every 8 connection(s)
I20260923 22:08:35.809252 12305 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:35.809983 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 12305
I20260923 22:08:35.810081 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:35.810336 2003 external_mini_cluster.cc:1428] Setting key 8e9f57e78cacd95f13accfbafd575353
Sep 23 22:08:35 dist-test-slave-b5jz krb5kdc[12198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201315, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260923 22:08:35.820204 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:35.809938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:35439 (local address 127.1.244.254:35633)
0923 22:08:35.810021 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:35.810023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:35.810246 (+ 223us) server_negotiation.cc:407] Connection header received
0923 22:08:35.810890 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:35.810891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:35.810901 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:35.810981 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:35.812326 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.813132 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.814507 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.814713 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.814771 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.817258 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:35.817260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:35.817260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:35.817268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:35.818759 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:35.819185 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:35.819186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:35.819186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:35.819250 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:35.819522 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:35.819523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:35.819523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:35.819671 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:35.819747 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:35.820059 (+ 312us) server_negotiation.cc:299] Negotiation successful
0923 22:08:35.820134 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 22:08:35.820370 12436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:35.809906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35633 (local address 127.1.244.193:35439)
0923 22:08:35.810088 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:35.810136 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:35.810781 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:35.811073 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:35.811078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:35.811942 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:35.812227 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.812233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.813231 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.813231 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.814416 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.814419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.814810 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.814812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.814983 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.815496 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:35.815508 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:35.817141 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:35.818866 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:35.818873 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:35.818875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:35.819102 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:35.819327 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:35.819328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:35.819328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:35.819435 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:35.819753 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:35.819759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:35.819918 (+ 159us) client_negotiation.cc:769] Sending connection context
0923 22:08:35.820033 (+ 115us) client_negotiation.cc:240] Negotiation successful
0923 22:08:35.820172 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260923 22:08:35.820885 12434 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35633
I20260923 22:08:35.821000 12434 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:35.821192 12434 heartbeater.cc:511] Master 127.1.244.254:35633 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
I20260923 22:08:35.821761 12243 ts_manager.cc:194] Registered new tserver with Master: c29337207e2c4b3aa39f7340278a8271 (127.1.244.193:36577)
I20260923 22:08:35.822677 12243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:35439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:08:35.834954 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35633
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:35.914391 12439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:35.914582 12439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:35.914604 12439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:35.916261 12439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:35.916316 12439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:35.916343 12439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:08:35.917723 12439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:35.917900 12439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:35.918062 12439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:35.922875 12439 server_base.cc:1034] running on GCE node
W20260923 22:08:35.922816 12444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:35.922847 12447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:35.922837 12445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:35.923218 12439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:35.923413 12439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:35.924551 12439 hybrid_clock.cc:648] HybridClock initialized: now 1790201315924535 us; error 33 us; skew 500 ppm
Sep 23 22:08:35 dist-test-slave-b5jz krb5kdc[12198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201315, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:35.926846 12439 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:08:35.927244 12439 webserver.cc:460] Webserver started at http://127.1.244.194:40595/ using document root <none> and password file <none>
I20260923 22:08:35.927465 12439 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:35.927533 12439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:35.927690 12439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:35.928665 12439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "442ad95163f3473f9431d542ea69861f"
format_stamp: "Formatted at 2026-09-23 22:08:35 on dist-test-slave-b5jz"
server_key: "12a7669dea00b45594f42c98148cf5f3"
server_key_iv: "1bf0d416d8a88a3b17e83bbe2cd4aa9a"
server_key_version: "encryptionkey@0"
I20260923 22:08:35.929008 12439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "442ad95163f3473f9431d542ea69861f"
format_stamp: "Formatted at 2026-09-23 22:08:35 on dist-test-slave-b5jz"
server_key: "12a7669dea00b45594f42c98148cf5f3"
server_key_iv: "1bf0d416d8a88a3b17e83bbe2cd4aa9a"
server_key_version: "encryptionkey@0"
I20260923 22:08:35.930241 12439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:35.930938 12454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:35.931100 12439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:35.931207 12439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "442ad95163f3473f9431d542ea69861f"
format_stamp: "Formatted at 2026-09-23 22:08:35 on dist-test-slave-b5jz"
server_key: "12a7669dea00b45594f42c98148cf5f3"
server_key_iv: "1bf0d416d8a88a3b17e83bbe2cd4aa9a"
server_key_version: "encryptionkey@0"
I20260923 22:08:35.931283 12439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:35.943816 12439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:35.945506 12439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:35.945672 12439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:35.945879 12439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:35.946178 12439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:35.946235 12439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:35.946275 12439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:35.946303 12439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:35.951934 12439 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:39697
I20260923 22:08:35.951974 12567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:39697 every 8 connection(s)
I20260923 22:08:35.952286 12439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:08:35 dist-test-slave-b5jz krb5kdc[12198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201315, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:35.959860 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 12439
I20260923 22:08:35.959983 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:35.960196 2003 external_mini_cluster.cc:1428] Setting key 388d4cb7c02a9e7fbede06b23ea6dfd9
I20260923 22:08:35.961254 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:35.952961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:41007 (local address 127.1.244.254:35633)
0923 22:08:35.953023 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:35.953025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:35.953248 (+ 223us) server_negotiation.cc:407] Connection header received
0923 22:08:35.953728 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:35.953730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:35.953739 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:35.953795 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:35.954725 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.955219 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.956234 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.956415 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.956466 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.958462 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:35.958464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:35.958465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:35.958473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:35.959791 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:35.960243 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:35.960245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:35.960245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:35.960309 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:35.960617 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:35.960617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:35.960618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:35.960720 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:35.960804 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:35.961115 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 22:08:35.961190 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:08:35.961550 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:35.952934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35633 (local address 127.1.244.194:41007)
0923 22:08:35.953106 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:35.953146 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:35.953634 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:35.953883 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:35.953887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:35.954437 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:35.954621 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.954625 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.955297 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.955297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.956160 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.956162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.956484 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.956485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.956585 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.957023 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:35.957031 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:35.958356 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:35.959875 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:35.959880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:35.959882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:35.960170 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:35.960397 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:35.960398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:35.960399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:35.960555 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:35.960797 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:35.960804 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:35.960962 (+ 158us) client_negotiation.cc:769] Sending connection context
0923 22:08:35.961088 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 22:08:35.961272 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260923 22:08:35.962050 12568 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35633
I20260923 22:08:35.962139 12568 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:35.962350 12568 heartbeater.cc:511] Master 127.1.244.254:35633 requested a full tablet report, sending...
I20260923 22:08:35.962762 12243 ts_manager.cc:194] Registered new tserver with Master: 442ad95163f3473f9431d542ea69861f (127.1.244.194:39697)
I20260923 22:08:35.963276 12243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:41007
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:08:35.985212 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35633
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:36.064711 12573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:36.064950 12573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:36.064988 12573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:36.066174 12573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:36.066243 12573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:36.066290 12573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:08:36.067762 12573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:08:36.068018 12573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:36.068245 12573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:36.073261 12578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:36.073274 12579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:36.073320 12573 server_base.cc:1034] running on GCE node
W20260923 22:08:36.073274 12581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:36.073654 12573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:36.073881 12573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:36.075033 12573 hybrid_clock.cc:648] HybridClock initialized: now 1790201316075021 us; error 25 us; skew 500 ppm
Sep 23 22:08:36 dist-test-slave-b5jz krb5kdc[12198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:36.077484 12573 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:08:36.077901 12573 webserver.cc:460] Webserver started at http://127.1.244.195:35217/ using document root <none> and password file <none>
I20260923 22:08:36.078117 12573 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:36.078187 12573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:36.078336 12573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:36.079331 12573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "8a8712898f024235b9e61538fe966dcb"
format_stamp: "Formatted at 2026-09-23 22:08:36 on dist-test-slave-b5jz"
server_key: "d8482526e1cc4cb2ef14f137e8a6c551"
server_key_iv: "22c315a70208b57bd7cae2c9f6297c2b"
server_key_version: "encryptionkey@0"
I20260923 22:08:36.079722 12573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "8a8712898f024235b9e61538fe966dcb"
format_stamp: "Formatted at 2026-09-23 22:08:36 on dist-test-slave-b5jz"
server_key: "d8482526e1cc4cb2ef14f137e8a6c551"
server_key_iv: "22c315a70208b57bd7cae2c9f6297c2b"
server_key_version: "encryptionkey@0"
I20260923 22:08:36.080946 12573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:36.081581 12588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:36.081756 12573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:36.081851 12573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "8a8712898f024235b9e61538fe966dcb"
format_stamp: "Formatted at 2026-09-23 22:08:36 on dist-test-slave-b5jz"
server_key: "d8482526e1cc4cb2ef14f137e8a6c551"
server_key_iv: "22c315a70208b57bd7cae2c9f6297c2b"
server_key_version: "encryptionkey@0"
I20260923 22:08:36.081933 12573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:36.101298 12573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:36.103029 12573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:36.103195 12573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:36.103425 12573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:36.103752 12573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:36.103807 12573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:36.103842 12573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:36.103870 12573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:36.109429 12573 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:34321
I20260923 22:08:36.109522 12701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:34321 every 8 connection(s)
I20260923 22:08:36.109813 12573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:36.110016 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 12573
I20260923 22:08:36.110157 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:36.110387 2003 external_mini_cluster.cc:1428] Setting key f2620f0ccbe66698c53edb1dc28cef7b
Sep 23 22:08:36 dist-test-slave-b5jz krb5kdc[12198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:08:36.118724 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.110537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:55525 (local address 127.1.244.254:35633)
0923 22:08:36.110589 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.110592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.110933 (+ 341us) server_negotiation.cc:407] Connection header received
0923 22:08:36.111537 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.111537 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.111593 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.111674 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:36.112614 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.113098 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.114010 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.114181 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.114229 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.116390 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:36.116391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:36.116391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:36.116397 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:36.117593 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:36.117945 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:36.117946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:36.117946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:36.117996 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:36.118215 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:36.118216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:36.118216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:36.118313 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:36.118362 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:36.118582 (+ 220us) server_negotiation.cc:299] Negotiation successful
0923 22:08:36.118659 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:08:36.118904 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.110528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35633 (local address 127.1.244.195:55525)
0923 22:08:36.110777 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.110843 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:36.111451 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:36.111773 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:36.111778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:36.112342 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:36.112539 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.112543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.113174 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.113175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.113940 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.113942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.114247 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.114248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.114341 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.114877 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:36.114894 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:36.116312 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:36.117666 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:36.117671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:36.117673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:36.117881 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:36.118053 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:36.118054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:36.118054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:36.118155 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:36.118366 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:36.118370 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:36.118479 (+ 109us) client_negotiation.cc:769] Sending connection context
0923 22:08:36.118558 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:08:36.118677 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":95,"threads_started":1}
I20260923 22:08:36.119449 12702 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35633
I20260923 22:08:36.119582 12702 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:36.119774 12702 heartbeater.cc:511] Master 127.1.244.254:35633 requested a full tablet report, sending...
I20260923 22:08:36.120160 12243 ts_manager.cc:194] Registered new tserver with Master: 8a8712898f024235b9e61538fe966dcb (127.1.244.195:34321)
I20260923 22:08:36.120548 12243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:55525
I20260923 22:08:36.122200 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:08:36.128845 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.123400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.1.244.254:35633)
0923 22:08:36.123457 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.123458 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.123586 (+ 128us) server_negotiation.cc:407] Connection header received
0923 22:08:36.123657 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.123657 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.123665 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.123712 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:36.124244 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.124685 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.125394 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.125564 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.125609 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.126712 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:36.126713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:36.126713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:36.126717 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:36.127903 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:36.128204 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:36.128205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:36.128205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:36.128252 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:36.128407 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:36.128407 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:36.128407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:36.128527 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:36.128601 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:36.128703 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 22:08:36.128755 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:08:36.130123 12243 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:36.131659 12243 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:36.138245 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.133880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:34321 (local address 127.0.0.1:47464)
0923 22:08:36.134629 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.134639 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:36.134694 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:36.134851 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:36.134851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:36.134853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:36.134899 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.134900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.135941 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.135942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.137227 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.137229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.137919 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.137920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.138005 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.138018 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:08:36.138106 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 22:08:36.138158 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":62,"threads_started":1}
I20260923 22:08:36.138368 12714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.134047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.1.244.194:39697)
0923 22:08:36.134224 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.134229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.134458 (+ 229us) server_negotiation.cc:407] Connection header received
0923 22:08:36.134635 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.134638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.134656 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.134726 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:36.134948 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.136045 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.137368 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.138034 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.138098 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.138145 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:36.138246 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 22:08:36.138288 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260923 22:08:36.138417 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.134068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47464 (local address 127.1.244.195:34321)
0923 22:08:36.134287 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.134291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.134687 (+ 396us) server_negotiation.cc:407] Connection header received
0923 22:08:36.134759 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.134761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.134776 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.134852 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:36.134997 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.135833 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.137326 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.137815 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.137915 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.137970 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:36.138296 (+ 326us) server_negotiation.cc:299] Negotiation successful
0923 22:08:36.138347 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":89,"threads_started":1}
I20260923 22:08:36.138664 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.133854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:39697 (local address 127.0.0.1:58968)
0923 22:08:36.134400 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.134409 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:36.134458 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:36.134806 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:36.134806 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:36.134808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:36.134853 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.134854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.136161 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.136162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.137294 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.137296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.138107 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.138108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.138169 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.138172 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:36.138220 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:08:36.138253 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":42,"threads_started":1}
I20260923 22:08:36.138367 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.133830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36577 (local address 127.0.0.1:35430)
0923 22:08:36.134184 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.134227 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:36.134300 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:36.134560 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:36.134563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:36.134566 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:36.134636 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.134638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.136190 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.136192 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.137164 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.137167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.138145 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.138146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.138211 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.138216 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:08:36.138263 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:08:36.138301 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":68,"threads_started":1}
I20260923 22:08:36.139268 12636 tablet_service.cc:1467] Processing CreateTablet for tablet 0ca6acb39be34e6dbf728f94ab89c1de (DEFAULT_TABLE table=test-table [id=8d20aa37adfe4f998d72be0efb519968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:36.139268 12502 tablet_service.cc:1467] Processing CreateTablet for tablet 0ca6acb39be34e6dbf728f94ab89c1de (DEFAULT_TABLE table=test-table [id=8d20aa37adfe4f998d72be0efb519968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:36.139626 12636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ca6acb39be34e6dbf728f94ab89c1de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:36.139626 12502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ca6acb39be34e6dbf728f94ab89c1de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:36.139647 12713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.133957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35430 (local address 127.1.244.193:36577)
0923 22:08:36.134104 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.134109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.134283 (+ 174us) server_negotiation.cc:407] Connection header received
0923 22:08:36.134382 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.134384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.134400 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.134476 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:36.134900 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.136085 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.137255 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.137968 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.138041 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.138109 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:36.138437 (+ 328us) server_negotiation.cc:299] Negotiation successful
0923 22:08:36.138493 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260923 22:08:36.142864 12716 tablet_bootstrap.cc:492] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: Bootstrap starting.
I20260923 22:08:36.143325 12368 tablet_service.cc:1467] Processing CreateTablet for tablet 0ca6acb39be34e6dbf728f94ab89c1de (DEFAULT_TABLE table=test-table [id=8d20aa37adfe4f998d72be0efb519968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:36.143537 12716 tablet_bootstrap.cc:654] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:36.143647 12368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ca6acb39be34e6dbf728f94ab89c1de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:36.144069 12716 log.cc:824] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:36.144364 12717 tablet_bootstrap.cc:492] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f: Bootstrap starting.
I20260923 22:08:36.144941 12717 tablet_bootstrap.cc:654] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:36.145464 12717 log.cc:824] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:36.145922 12716 tablet_bootstrap.cc:492] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: No bootstrap required, opened a new log
I20260923 22:08:36.145998 12716 ts_tablet_manager.cc:1397] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:08:36.146191 12717 tablet_bootstrap.cc:492] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f: No bootstrap required, opened a new log
I20260923 22:08:36.146255 12717 ts_tablet_manager.cc:1397] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:36.146643 12720 tablet_bootstrap.cc:492] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271: Bootstrap starting.
I20260923 22:08:36.147207 12720 tablet_bootstrap.cc:654] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:36.147782 12720 log.cc:824] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:36.147981 12716 raft_consensus.cc:348] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } }
I20260923 22:08:36.148152 12716 raft_consensus.cc:374] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:36.148229 12716 raft_consensus.cc:729] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a8712898f024235b9e61538fe966dcb, State: Initialized, Role: FOLLOWER
I20260923 22:08:36.148167 12717 raft_consensus.cc:348] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } }
I20260923 22:08:36.148340 12717 raft_consensus.cc:374] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:36.148397 12717 raft_consensus.cc:729] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442ad95163f3473f9431d542ea69861f, State: Initialized, Role: FOLLOWER
I20260923 22:08:36.148378 12716 consensus_queue.cc:260] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } }
I20260923 22:08:36.148480 12720 tablet_bootstrap.cc:492] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271: No bootstrap required, opened a new log
I20260923 22:08:36.148548 12720 ts_tablet_manager.cc:1397] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:08:36.148511 12717 consensus_queue.cc:260] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } }
I20260923 22:08:36.148686 12702 heartbeater.cc:503] Master 127.1.244.254:35633 was elected leader, sending a full tablet report...
I20260923 22:08:36.148689 12716 ts_tablet_manager.cc:1428] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:08:36.148753 12717 ts_tablet_manager.cc:1428] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:36.148845 12568 heartbeater.cc:503] Master 127.1.244.254:35633 was elected leader, sending a full tablet report...
I20260923 22:08:36.151892 12720 raft_consensus.cc:348] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } }
I20260923 22:08:36.152076 12720 raft_consensus.cc:374] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:36.152119 12720 raft_consensus.cc:729] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29337207e2c4b3aa39f7340278a8271, State: Initialized, Role: FOLLOWER
I20260923 22:08:36.152230 12720 consensus_queue.cc:260] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } }
I20260923 22:08:36.152588 12720 ts_tablet_manager.cc:1428] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260923 22:08:36.152616 12434 heartbeater.cc:503] Master 127.1.244.254:35633 was elected leader, sending a full tablet report...
W20260923 22:08:36.202903 12569 tablet.cc:2369] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:08:36.309947 12435 tablet.cc:2369] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:36.319010 12722 raft_consensus.cc:482] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:36.319154 12722 raft_consensus.cc:504] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } }
I20260923 22:08:36.319599 12722 leader_election.cc:288] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c29337207e2c4b3aa39f7340278a8271 (127.1.244.193:36577), 442ad95163f3473f9431d542ea69861f (127.1.244.194:39697)
I20260923 22:08:36.322278 12713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.319773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:42983 (local address 127.1.244.193:36577)
0923 22:08:36.319814 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.319816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.319825 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:36.319916 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.319917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.319934 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.320003 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:36.320179 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.320665 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.321564 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.321919 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.322002 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.322010 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:36.322172 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 22:08:36.322208 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:08:36.322278 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.319715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36577 (local address 127.1.244.195:42983)
0923 22:08:36.319767 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.319779 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:36.319819 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:36.320005 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:36.320006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:36.320008 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:36.320085 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.320086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.320759 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.320760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.321463 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.321465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.321999 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.322000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.322078 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.322081 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:36.322178 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 22:08:36.322215 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 22:08:36.322592 12714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.319834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:46517 (local address 127.1.244.194:39697)
0923 22:08:36.319880 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.319882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.319994 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:08:36.320058 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.320059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.320069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.320124 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:36.320287 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.320847 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.321758 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.322136 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.322218 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.322226 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:36.322431 (+ 205us) server_negotiation.cc:299] Negotiation successful
0923 22:08:36.322485 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:08:36.322587 12388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ca6acb39be34e6dbf728f94ab89c1de" candidate_uuid: "8a8712898f024235b9e61538fe966dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29337207e2c4b3aa39f7340278a8271" is_pre_election: true
I20260923 22:08:36.322758 12388 raft_consensus.cc:2454] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a8712898f024235b9e61538fe966dcb in term 0.
I20260923 22:08:36.322801 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.319748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:39697 (local address 127.1.244.195:46517)
0923 22:08:36.319920 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.319929 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:36.320004 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:36.320132 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:36.320133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:36.320134 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:36.320211 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.320212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.320927 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.320928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.321663 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.321665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.322229 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.322231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.322317 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.322320 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:36.322685 (+ 365us) client_negotiation.cc:240] Negotiation successful
0923 22:08:36.322733 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260923 22:08:36.323006 12591 leader_election.cc:302] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a8712898f024235b9e61538fe966dcb, c29337207e2c4b3aa39f7340278a8271; no voters:
I20260923 22:08:36.323155 12722 raft_consensus.cc:2781] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:36.323144 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ca6acb39be34e6dbf728f94ab89c1de" candidate_uuid: "8a8712898f024235b9e61538fe966dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442ad95163f3473f9431d542ea69861f" is_pre_election: true
I20260923 22:08:36.323222 12722 raft_consensus.cc:482] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:36.323247 12722 raft_consensus.cc:3037] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:36.323284 12522 raft_consensus.cc:2454] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a8712898f024235b9e61538fe966dcb in term 0.
I20260923 22:08:36.324118 12722 raft_consensus.cc:504] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } }
I20260923 22:08:36.324249 12722 leader_election.cc:288] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [CANDIDATE]: Term 1 election: Requested vote from peers c29337207e2c4b3aa39f7340278a8271 (127.1.244.193:36577), 442ad95163f3473f9431d542ea69861f (127.1.244.194:39697)
I20260923 22:08:36.324437 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ca6acb39be34e6dbf728f94ab89c1de" candidate_uuid: "8a8712898f024235b9e61538fe966dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442ad95163f3473f9431d542ea69861f"
I20260923 22:08:36.324481 12388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ca6acb39be34e6dbf728f94ab89c1de" candidate_uuid: "8a8712898f024235b9e61538fe966dcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29337207e2c4b3aa39f7340278a8271"
I20260923 22:08:36.324540 12522 raft_consensus.cc:3037] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:36.324538 12388 raft_consensus.cc:3037] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:36.325223 12388 raft_consensus.cc:2454] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a8712898f024235b9e61538fe966dcb in term 1.
I20260923 22:08:36.325317 12522 raft_consensus.cc:2454] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a8712898f024235b9e61538fe966dcb in term 1.
I20260923 22:08:36.325384 12591 leader_election.cc:302] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a8712898f024235b9e61538fe966dcb, c29337207e2c4b3aa39f7340278a8271; no voters:
I20260923 22:08:36.325469 12722 raft_consensus.cc:2781] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:36.325601 12722 raft_consensus.cc:686] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 1 LEADER]: Becoming Leader. State: Replica: 8a8712898f024235b9e61538fe966dcb, State: Running, Role: LEADER
I20260923 22:08:36.325708 12722 consensus_queue.cc:237] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } }
I20260923 22:08:36.326462 12241 catalog_manager.cc:5495] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a8712898f024235b9e61538fe966dcb (127.1.244.195). New cstate: current_term: 1 leader_uuid: "8a8712898f024235b9e61538fe966dcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a8712898f024235b9e61538fe966dcb" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34321 } health_report { overall_health: HEALTHY } } }
W20260923 22:08:36.360591 12703 tablet.cc:2369] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:36.364152 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.361979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.1.244.195:34321)
0923 22:08:36.362040 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.362041 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.362052 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:08:36.362102 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.362102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.362118 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.362178 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:36.362297 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.362755 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.363436 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.363634 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.363683 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.363822 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:36.363914 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:36.363976 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:36.364003 (+ 27us) server_negotiation.cc:299] Negotiation successful
0923 22:08:36.364020 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:08:36.365881 12388 raft_consensus.cc:1261] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [term 1 FOLLOWER]: Refusing update from remote peer 8a8712898f024235b9e61538fe966dcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:36.365882 12522 raft_consensus.cc:1261] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [term 1 FOLLOWER]: Refusing update from remote peer 8a8712898f024235b9e61538fe966dcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:36.366232 12722 consensus_queue.cc:1035] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29337207e2c4b3aa39f7340278a8271" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:36.366364 12726 consensus_queue.cc:1035] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "442ad95163f3473f9431d542ea69861f" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:36.368815 12730 mvcc.cc:205] Tried to move back new op lower bound from 7332664591832047616 to 7332664591670743040. Current Snapshot: MvccSnapshot[applied={T|T < 7332664591832047616}]
I20260923 22:08:36.371351 12242 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:36.371452 12242 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:36.372224 12242 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89d9ed4bdfd04d0b921e58f3854169de: Sending DeleteTablet for 3 replicas of tablet 0ca6acb39be34e6dbf728f94ab89c1de
I20260923 22:08:36.372637 12368 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ca6acb39be34e6dbf728f94ab89c1de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:36 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:35430
I20260923 22:08:36.372695 12502 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ca6acb39be34e6dbf728f94ab89c1de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:36 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:58968
I20260923 22:08:36.372855 12738 tablet_replica.cc:330] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271: stopping tablet replica
I20260923 22:08:36.372856 12636 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ca6acb39be34e6dbf728f94ab89c1de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:36 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:47464
I20260923 22:08:36.372987 12738 raft_consensus.cc:2229] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:36.373041 12740 tablet_replica.cc:330] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: stopping tablet replica
I20260923 22:08:36.373159 12738 raft_consensus.cc:2258] T 0ca6acb39be34e6dbf728f94ab89c1de P c29337207e2c4b3aa39f7340278a8271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:36.373168 12740 raft_consensus.cc:2229] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:36.373175 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 12305
I20260923 22:08:36.373445 12740 raft_consensus.cc:2258] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:36.373955 12740 ts_tablet_manager.cc:1905] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:36.375677 12740 ts_tablet_manager.cc:1918] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:36.375758 12740 log.cc:1196] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/0ca6acb39be34e6dbf728f94ab89c1de
I20260923 22:08:36.376032 12740 ts_tablet_manager.cc:1939] T 0ca6acb39be34e6dbf728f94ab89c1de P 8a8712898f024235b9e61538fe966dcb: Deleting consensus metadata
I20260923 22:08:36.376415 12227 catalog_manager.cc:4841] TS 8a8712898f024235b9e61538fe966dcb (127.1.244.195:34321): tablet 0ca6acb39be34e6dbf728f94ab89c1de (table test-table [id=8d20aa37adfe4f998d72be0efb519968]) successfully deleted
I20260923 22:08:36.379743 12739 tablet_replica.cc:330] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f: stopping tablet replica
I20260923 22:08:36.379927 12739 raft_consensus.cc:2229] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:36.380053 12739 raft_consensus.cc:2258] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:36.380470 12739 ts_tablet_manager.cc:1905] T 0ca6acb39be34e6dbf728f94ab89c1de P 442ad95163f3473f9431d542ea69861f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:08:36.381201 12229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:36577 (error 108)
I20260923 22:08:36.381735 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 12439
I20260923 22:08:36.381942 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.381823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36577 (local address 127.0.0.1:35440)
0923 22:08:36.381873 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.381907 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 22:08:36.382045 12229 catalog_manager.cc:4584] TS c29337207e2c4b3aa39f7340278a8271 (127.1.244.193:36577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ca6acb39be34e6dbf728f94ab89c1de: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36577: connect: Connection refused (error 111)
I20260923 22:08:36.387127 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 12573
I20260923 22:08:36.387136 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.387037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:39697 (local address 127.0.0.1:58978)
0923 22:08:36.387084 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.387105 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.194:39697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:08:36.391652 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 12211
2026-09-23T22:08:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2052 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:36 dist-test-slave-b5jz krb5kdc[12744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:36 dist-test-slave-b5jz krb5kdc[12744](info): set up 2 sockets
Sep 23 22:08:36 dist-test-slave-b5jz krb5kdc[12744](info): commencing operation
krb5kdc: starting...
W20260923 22:08:37.424247 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:08:37 dist-test-slave-b5jz krb5kdc[12744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/rangeradmin_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/rangeradmin_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/rangerlookup_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/rangerlookup_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
I20260923 22:08:37.574340 2003 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6wdnYN/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/postgres -l logfile start
2026-09-23 22:08:39.849 UTC [12774] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 22:08:39.849 UTC [12774] LOG: listening on IPv4 address "127.1.244.212", port 46107
2026-09-23 22:08:39.852 UTC [12774] LOG: listening on Unix socket "/tmp/.s.PGSQL.46107"
2026-09-23 22:08:39.886 UTC [12777] LOG: database system was shut down at 2026-09-23 22:08:38 UTC
2026-09-23 22:08:39.890 UTC [12774] LOG: database system is ready to accept connections
I20260923 22:08:40.824302 2003 mini_postgres.cc:96] Postgres bound to 46107
2026-09-23 22:08:40.830 UTC [12785] FATAL: database "slave" does not exist
127.1.244.212:46107 - accepting connections
I20260923 22:08:40.830652 2003 mini_ranger.cc:161] Starting Ranger...
I20260923 22:08:40.844590 2003 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 22:08:41.276724 2003 mini_ranger.cc:87] Created ranger Postgres database
I20260923 22:08:41.276811 2003 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-admin
2026-09-23 22:08:41,550 [I] DB FLAVOR :POSTGRES
2026-09-23 22:08:41,550 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:08:41,551 [I] Checking connection..
2026-09-23 22:08:41,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 22:08:41,787 [I] Checking connection passed.
2026-09-23 22:08:41,788 [I] --------- Verifying version history table ---------
2026-09-23 22:08:41,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:08:42,025 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:08:42,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:08:42,284 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:08:42,284 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 22:08:42,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 22:08:42.518 UTC [12849] WARNING: there is no transaction in progress
2026-09-23 22:08:42,524 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 22:08:42,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:08:42,753 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 22:08:42,754 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:08:42,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 22:08:43,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 22:08:43,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-b5jz.c.gcp-upstream.internal','N') ;"
2026-09-23 22:08:43,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:08:43,763 [I] Table x_portal_user does not exist in database ranger
2026-09-23 22:08:43,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:08:44,006 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 22:08:44,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:08:44,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:08:44,480 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 22:08:44,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 22:08:45.402 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.416 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.430 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.444 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.459 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.475 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.525 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.531 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.544 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.553 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.562 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.574 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.583 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.589 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.595 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:45.601 UTC [12957] WARNING: there is no transaction in progress
2026-09-23 22:08:46,168 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 22:08:46,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 22:08:46,400 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 22:08:46,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:08:46,658 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 22:08:46,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:08:46,891 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 22:08:46,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:08:47,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:08:47,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-b5jz.c.gcp-upstream.internal';"
2026-09-23 22:08:47,560 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 22:08:47,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 22:08:47,818 [I] DB_PATCHES have already been applied
I20260923 22:08:47.823526 2003 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-admin:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6wdnYN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 22:08:47.823648 2003 mini_ranger.cc:193] Using host: 127.1.244.212
I20260923 22:08:47.826510 2003 mini_ranger.cc:227] Ranger admin URL: http://127.1.244.212:59405
Sep 23, 2026 10:08:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:08:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:08:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 10:08:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 10:08:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 10:08:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 10:08:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 22:08:48.917474 2003 mini_ranger.cc:160] Time spent starting Ranger: real 8.087s user 0.000s sys 0.008s
Sep 23, 2026 10:09:08 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 10:09:08 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 23, 2026 10:09:08 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 10:09:08 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 10:09:09 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 22:09:10.005962 2003 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/rangerkms_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/rangerkms_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.244.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.244.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 22:09:10 dist-test-slave-b5jz krb5kdc[12744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201350, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 22:09:10.093284 2003 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 22:09:10.102428 2003 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 22:09:10.965838 2003 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 22:09:10.965927 2003 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-kms
I20260923 22:09:10.965945 2003 mini_ranger_kms.cc:227] Using postgres at 127.1.244.212:46107
2026-09-23 22:09:11,144 [I] DB FLAVOR :POSTGRES
2026-09-23 22:09:11,145 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:09:11,145 [I] Checking connection
2026-09-23 22:09:11,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 22:09:11,411 [I] connection success
2026-09-23 22:09:11,411 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 22:09:11,411 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 22:09:11,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 22:09:11,651 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 22:09:11,652 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:09:11,652 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 22:09:11,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:46107/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 22:09:11,923 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 22:09:12.427281 2003 mini_ranger_kms.cc:310] Created kms service
I20260923 22:09:12.597963 2003 mini_ranger_kms.cc:326] Created kudu user
I20260923 22:09:12.640674 2003 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 22:09:12 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201350, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:09:13.042579 2003 mini_ranger_kms.cc:384] Added ranger policy
I20260923 22:09:13.042701 2003 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-kms:/tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 22:09:13.042721 2003 mini_ranger_kms.cc:240] Using host: 127.1.244.212
I20260923 22:09:13.046627 2003 mini_ranger_kms.cc:276] Ranger KMS PID: 13137
I20260923 22:09:13.046753 2003 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.244.212:60501
Sep 23, 2026 10:09:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:09:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:09:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 10:09:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 10:09:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 10:09:14 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 22:09:15.239928 2003 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.147s user 0.000s sys 0.161s
I20260923 22:09:15.240036 2003 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 22:09:15.240115 2003 mini_ranger_kms.cc:401] 127.1.244.212:60501/kms/v1/keys
Sep 23 22:09:15 dist-test-slave-b5jz krb5kdc[12744](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201355, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 22:09:15 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201355, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Sep 23 22:09:16 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201355, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Sep 23 22:09:18 dist-test-slave-b5jz krb5kdc[12744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:09:18.550983 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37241
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:37241
--ranger_config_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:60501/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:09:18.634362 13188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:09:18.634642 13188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:09:18.634681 13188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:09:18.635979 13188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:09:18.636045 13188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:09:18.636080 13188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:09:18.636108 13188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:09:18.636135 13188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:09:18.637781 13188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.244.212:60501/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:37241
--ranger_config_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:09:18.638092 13188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:09:18.638335 13188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:09:18.643725 13193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:18.643734 13194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:18.643755 13188 server_base.cc:1034] running on GCE node
W20260923 22:09:18.643738 13196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:18.644292 13188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:09:18.644671 13188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:09:18.645828 13188 hybrid_clock.cc:648] HybridClock initialized: now 1790201358645806 us; error 35 us; skew 500 ppm
Sep 23 22:09:18 dist-test-slave-b5jz krb5kdc[12744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201358, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:18.652727 13188 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:09:18.653386 13188 webserver.cc:460] Webserver started at http://127.1.244.254:37503/ using document root <none> and password file <none>
I20260923 22:09:18.653694 13188 fs_manager.cc:359] Metadata directory not provided
I20260923 22:09:18.653752 13188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:09:18.653899 13188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:09:18 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201358, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:09:18.707094 13188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "1ef97230077549caadf0ea0e9ac299f7"
format_stamp: "Formatted at 2026-09-23 22:09:18 on dist-test-slave-b5jz"
server_key: "c8520cda2d6e22dba11918a14b746be0"
server_key_iv: "eb47f296f8f307a962e37f952aaa2e59"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:18.707535 13188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "1ef97230077549caadf0ea0e9ac299f7"
format_stamp: "Formatted at 2026-09-23 22:09:18 on dist-test-slave-b5jz"
server_key: "c8520cda2d6e22dba11918a14b746be0"
server_key_iv: "eb47f296f8f307a962e37f952aaa2e59"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:18.709090 13188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:09:18.725212 13203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:18.725737 13188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:09:18.725853 13188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "1ef97230077549caadf0ea0e9ac299f7"
format_stamp: "Formatted at 2026-09-23 22:09:18 on dist-test-slave-b5jz"
server_key: "c8520cda2d6e22dba11918a14b746be0"
server_key_iv: "eb47f296f8f307a962e37f952aaa2e59"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:18.725955 13188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:09:18.741653 13188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:09:18.744132 13188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:09:18.744304 13188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:09:18.749300 13188 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:37241
I20260923 22:09:18.749346 13255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:37241 every 8 connection(s)
I20260923 22:09:18.749720 13188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:09:18.750434 13256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:09:18.753223 13256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7: Bootstrap starting.
I20260923 22:09:18.753862 13256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7: Neither blocks nor log segments found. Creating new log.
I20260923 22:09:18.754169 13256 log.cc:824] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7: Log is configured to *not* fsync() on all Append() calls
I20260923 22:09:18.754863 13256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7: No bootstrap required, opened a new log
I20260923 22:09:18.756968 13256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef97230077549caadf0ea0e9ac299f7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37241 } }
I20260923 22:09:18.757119 13256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:09:18.757151 13256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef97230077549caadf0ea0e9ac299f7, State: Initialized, Role: FOLLOWER
I20260923 22:09:18.757319 13256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef97230077549caadf0ea0e9ac299f7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37241 } }
I20260923 22:09:18.757406 13256 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:09:18.757445 13256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:09:18.757512 13256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:09:18.757508 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 13188
I20260923 22:09:18.757602 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:09:18.757819 2003 external_mini_cluster.cc:1428] Setting key e27826f0074408f18b33328b615e41ca
I20260923 22:09:18.758239 13256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef97230077549caadf0ea0e9ac299f7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37241 } }
I20260923 22:09:18.758404 13256 leader_election.cc:302] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef97230077549caadf0ea0e9ac299f7; no voters:
I20260923 22:09:18.758642 13256 leader_election.cc:288] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:09:18.758674 13261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:09:18.758879 13261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [term 1 LEADER]: Becoming Leader. State: Replica: 1ef97230077549caadf0ea0e9ac299f7, State: Running, Role: LEADER
I20260923 22:09:18.758966 13256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:09:18.759011 13261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef97230077549caadf0ea0e9ac299f7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37241 } }
I20260923 22:09:18.759405 13261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ef97230077549caadf0ea0e9ac299f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef97230077549caadf0ea0e9ac299f7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37241 } } }
I20260923 22:09:18.759605 13261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:09:18.759806 13261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ef97230077549caadf0ea0e9ac299f7. Latest consensus state: current_term: 1 leader_uuid: "1ef97230077549caadf0ea0e9ac299f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef97230077549caadf0ea0e9ac299f7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37241 } } }
I20260923 22:09:18.759907 13261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:09:18.760075 13270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:09:18.760634 13270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:09:18.763046 13270 catalog_manager.cc:1347] Generated new cluster ID: e75d75af19d64e71a2d2c63888b33dd0
I20260923 22:09:18.763149 13270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:09:18 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:09:18.767850 13256 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 22:09:18.771296 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:18.758407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.1.244.254:37241)
0923 22:09:18.758632 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:18.758643 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:18.758676 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:09:18.758858 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:18.758866 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:18.758937 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:18.759925 (+ 988us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:18.760037 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:18.761445 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:18.762534 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:18.763177 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:18.766372 (+ 3195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:18.766469 (+ 97us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:18.766472 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:18.766473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:18.766483 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:18.769141 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:18.769574 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:18.769577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:18.769577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:18.769683 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:18.769912 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:18.769913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:18.769913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:18.770248 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:18.770878 (+ 630us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:18.770952 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:09:18.771076 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":104,"threads_started":1}
I20260923 22:09:18.788385 13270 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:09:18.789278 13270 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:09:18.793915 13270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7: Generated new TSK 0
I20260923 22:09:18.794195 13270 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:09:19.551119 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:60501/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37241
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:09:19.674731 13301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:09:19.675019 13301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:09:19.675057 13301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:09:19.676805 13301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:09:19.676870 13301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:09:19.676911 13301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:09:19.679141 13301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.244.212:60501/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:09:19.679399 13301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:09:19.679716 13301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:09:19.686111 13306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:19.686120 13309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:19.686133 13307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:19.686956 13301 server_base.cc:1034] running on GCE node
I20260923 22:09:19.687188 13301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:09:19.687486 13301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:09:19.688696 13301 hybrid_clock.cc:648] HybridClock initialized: now 1790201359688682 us; error 31 us; skew 500 ppm
Sep 23 22:09:19 dist-test-slave-b5jz krb5kdc[12744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:19.691437 13301 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:09:19.691941 13301 webserver.cc:460] Webserver started at http://127.1.244.193:40735/ using document root <none> and password file <none>
I20260923 22:09:19.692214 13301 fs_manager.cc:359] Metadata directory not provided
I20260923 22:09:19.692294 13301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:09:19.692435 13301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:09:19 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:09:19.706772 13301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "3df6369f304744dc8fdec4be1f81783d"
format_stamp: "Formatted at 2026-09-23 22:09:19 on dist-test-slave-b5jz"
server_key: "8976fe80780c3b95a274ac4e0a4e3b5a"
server_key_iv: "30537683da5e1932afcacfd505cef4c0"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:19.707230 13301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "3df6369f304744dc8fdec4be1f81783d"
format_stamp: "Formatted at 2026-09-23 22:09:19 on dist-test-slave-b5jz"
server_key: "8976fe80780c3b95a274ac4e0a4e3b5a"
server_key_iv: "30537683da5e1932afcacfd505cef4c0"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:19.711262 13301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260923 22:09:19.727401 13316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:19.727710 13301 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260923 22:09:19.727824 13301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "3df6369f304744dc8fdec4be1f81783d"
format_stamp: "Formatted at 2026-09-23 22:09:19 on dist-test-slave-b5jz"
server_key: "8976fe80780c3b95a274ac4e0a4e3b5a"
server_key_iv: "30537683da5e1932afcacfd505cef4c0"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:19.727924 13301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:09:19.744243 13301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:09:19.746712 13301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:09:19.746927 13301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:09:19.747253 13301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:09:19.747686 13301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:09:19.747807 13301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:19.747905 13301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:09:19.747977 13301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:19.758136 13301 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:37029
I20260923 22:09:19.758713 13301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:09:19.761126 13431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:37029 every 8 connection(s)
I20260923 22:09:19.763988 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 13301
I20260923 22:09:19.764118 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:09:19.764339 2003 external_mini_cluster.cc:1428] Setting key a35cd4aa522611bf885e866420641170
Sep 23 22:09:19 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260923 22:09:19.776295 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:19.762150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53697 (local address 127.1.244.254:37241)
0923 22:09:19.763705 (+ 1555us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:19.763707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:19.763715 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:09:19.763743 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:19.763744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:19.763758 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:19.764578 (+ 820us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:19.766822 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:19.767668 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:19.769286 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:19.769577 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:19.769646 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:19.772509 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:19.772511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:19.772511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:19.772521 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:19.774549 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:19.775103 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:19.775105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:19.775105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:19.775169 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:19.775438 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:19.775439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:19.775439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:19.775637 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:19.775709 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:19.776105 (+ 396us) server_negotiation.cc:299] Negotiation successful
0923 22:09:19.776195 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1522,"thread_start_us":68,"threads_started":1}
I20260923 22:09:19.776461 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:19.761865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37241 (local address 127.1.244.193:53697)
0923 22:09:19.762413 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:19.762460 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:19.763486 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:19.764724 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:19.764730 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:19.764982 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:09:19.766707 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:19.766715 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:19.767767 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:19.767768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:19.769180 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:19.769184 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:19.769721 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:19.769722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:19.769891 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:19.770512 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:09:19.770523 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:09:19.772389 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:09:19.774666 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:19.774675 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:19.774678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:19.775008 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:19.775242 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:19.775243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:19.775243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:19.775372 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:19.775740 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:09:19.775748 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:09:19.775929 (+ 181us) client_negotiation.cc:769] Sending connection context
0923 22:09:19.776052 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 22:09:19.776211 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":74,"threads_started":1}
I20260923 22:09:19.777253 13432 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37241
I20260923 22:09:19.777379 13432 heartbeater.cc:464] Registering TS with master...
I20260923 22:09:19.777575 13432 heartbeater.cc:511] Master 127.1.244.254:37241 requested a full tablet report, sending...
I20260923 22:09:19.778199 13220 ts_manager.cc:194] Registered new tserver with Master: 3df6369f304744dc8fdec4be1f81783d (127.1.244.193:37029)
I20260923 22:09:19.778788 13220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:53697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:09:19.794947 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:60501/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37241
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:09:19.908597 13438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:09:19.908893 13438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:09:19.908937 13438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:09:19.910749 13438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:09:19.910842 13438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:09:19.910902 13438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:09:19.913242 13438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.244.212:60501/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:09:19.913655 13438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:09:19.913957 13438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:09:19.920433 13446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:19.920560 13445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:19.920446 13448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Sep 23 22:09:19 dist-test-slave-b5jz krb5kdc[12744](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:19.922394 13438 server_base.cc:1034] running on GCE node
I20260923 22:09:19.922623 13438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:09:19.922933 13438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:09:19.924189 13438 hybrid_clock.cc:648] HybridClock initialized: now 1790201359924172 us; error 37 us; skew 500 ppm
Sep 23 22:09:19 dist-test-slave-b5jz krb5kdc[12744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:19.927742 13438 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:09:19.928318 13438 webserver.cc:460] Webserver started at http://127.1.244.194:45187/ using document root <none> and password file <none>
I20260923 22:09:19.928608 13438 fs_manager.cc:359] Metadata directory not provided
I20260923 22:09:19.928736 13438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:09:19.928921 13438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:09:19 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:09:19.949451 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "494a20ed6a0a4869a66225ca1b6db383"
format_stamp: "Formatted at 2026-09-23 22:09:19 on dist-test-slave-b5jz"
server_key: "da64671b75c374bce09f89f80965fb6b"
server_key_iv: "87cff11398b5ba1961b7152830c94665"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:19.950039 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "494a20ed6a0a4869a66225ca1b6db383"
format_stamp: "Formatted at 2026-09-23 22:09:19 on dist-test-slave-b5jz"
server_key: "da64671b75c374bce09f89f80965fb6b"
server_key_iv: "87cff11398b5ba1961b7152830c94665"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:19.959096 13438 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.001s
I20260923 22:09:19.979724 13455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:19.979949 13438 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260923 22:09:19.980103 13438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "494a20ed6a0a4869a66225ca1b6db383"
format_stamp: "Formatted at 2026-09-23 22:09:19 on dist-test-slave-b5jz"
server_key: "da64671b75c374bce09f89f80965fb6b"
server_key_iv: "87cff11398b5ba1961b7152830c94665"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:19.980237 13438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:09:20.008807 13438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:09:20.011107 13438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:09:20.011278 13438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:09:20.011539 13438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:09:20.011965 13438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:09:20.012070 13438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:20.012146 13438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:09:20.012209 13438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:20.020421 13438 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:44093
I20260923 22:09:20.021595 13438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:09:20.022533 13568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:44093 every 8 connection(s)
I20260923 22:09:20.025701 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 13438
I20260923 22:09:20.026198 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:09:20.026412 2003 external_mini_cluster.cc:1428] Setting key f04e4d315fe95e96cab5a3d2234fd141
Sep 23 22:09:20 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:09:20.034413 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.022398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37241 (local address 127.1.244.194:52091)
0923 22:09:20.022814 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:20.022857 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:20.023505 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:20.023872 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:20.023877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:20.024139 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:09:20.024426 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.024432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.025464 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.025466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.027545 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.027548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.028160 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.028161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.028316 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.028910 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:09:20.028921 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:09:20.030656 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:09:20.032726 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:20.032735 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:20.032738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:20.033070 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:20.033287 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:20.033287 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:20.033288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:20.033419 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:20.033705 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:09:20.033710 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:09:20.033890 (+ 180us) client_negotiation.cc:769] Sending connection context
0923 22:09:20.033996 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:09:20.034157 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":84,"threads_started":1}
I20260923 22:09:20.035293 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.022382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:52091 (local address 127.1.244.254:37241)
0923 22:09:20.022436 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:20.022438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:20.022973 (+ 535us) server_negotiation.cc:407] Connection header received
0923 22:09:20.023675 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:20.023676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:20.023698 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:20.023785 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:20.024547 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.025348 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.027670 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.028021 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.028067 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.030789 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:20.030791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:20.030792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:20.030800 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:20.032615 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:20.033158 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:20.033159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:20.033160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:20.033221 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:20.033484 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:20.033485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:20.033485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:20.033624 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:20.033681 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:20.035086 (+ 1405us) server_negotiation.cc:299] Negotiation successful
0923 22:09:20.035180 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:09:20.035907 13569 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37241
I20260923 22:09:20.036010 13569 heartbeater.cc:464] Registering TS with master...
I20260923 22:09:20.036201 13569 heartbeater.cc:511] Master 127.1.244.254:37241 requested a full tablet report, sending...
I20260923 22:09:20.036576 13220 ts_manager.cc:194] Registered new tserver with Master: 494a20ed6a0a4869a66225ca1b6db383 (127.1.244.194:44093)
I20260923 22:09:20.037086 13220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:52091
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:09:20.065142 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:60501/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37241
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:09:20.187218 13575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:09:20.187603 13575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:09:20.187702 13575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:09:20.189527 13575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:09:20.189651 13575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:09:20.189746 13575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:09:20.192131 13575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.244.212:60501/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:09:20.192512 13575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:09:20.192827 13575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:09:20.199630 13583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:20.200232 13581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:20.200495 13580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:20.201045 13575 server_base.cc:1034] running on GCE node
I20260923 22:09:20.201288 13575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:09:20.201550 13575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:09:20.204829 13575 hybrid_clock.cc:648] HybridClock initialized: now 1790201360204815 us; error 27 us; skew 500 ppm
Sep 23 22:09:20 dist-test-slave-b5jz krb5kdc[12744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:20.207967 13575 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:09:20.208434 13575 webserver.cc:460] Webserver started at http://127.1.244.195:44521/ using document root <none> and password file <none>
I20260923 22:09:20.208674 13575 fs_manager.cc:359] Metadata directory not provided
I20260923 22:09:20.208746 13575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:09:20.208899 13575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:09:20 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:09:20.221004 13575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "2bf82557f0294aedb57e5685008a59c7"
format_stamp: "Formatted at 2026-09-23 22:09:20 on dist-test-slave-b5jz"
server_key: "b0bccbb99c59880ec0b3285e190b0c78"
server_key_iv: "96496b279d9370edd78c5754eb43879d"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:20.221472 13575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "2bf82557f0294aedb57e5685008a59c7"
format_stamp: "Formatted at 2026-09-23 22:09:20 on dist-test-slave-b5jz"
server_key: "b0bccbb99c59880ec0b3285e190b0c78"
server_key_iv: "96496b279d9370edd78c5754eb43879d"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:20.223356 13575 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 22:09:20.255666 13590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:20.255913 13575 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260923 22:09:20.256028 13575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "2bf82557f0294aedb57e5685008a59c7"
format_stamp: "Formatted at 2026-09-23 22:09:20 on dist-test-slave-b5jz"
server_key: "b0bccbb99c59880ec0b3285e190b0c78"
server_key_iv: "96496b279d9370edd78c5754eb43879d"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:20.256152 13575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:09:20.272886 13575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:09:20.275180 13575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:09:20.275354 13575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:09:20.275688 13575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:09:20.276067 13575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:09:20.276142 13575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:20.276193 13575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:09:20.276234 13575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:20.285867 13575 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:39115
I20260923 22:09:20.286358 13575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:09:20.291688 13703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:39115 every 8 connection(s)
I20260923 22:09:20.293071 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 13575
I20260923 22:09:20.293169 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:09:20.293370 2003 external_mini_cluster.cc:1428] Setting key 9a96e193b673a224ea99027433212652
Sep 23 22:09:20 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:09:20.302989 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.292612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:32779 (local address 127.1.244.254:37241)
0923 22:09:20.292689 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:20.292692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:20.292916 (+ 224us) server_negotiation.cc:407] Connection header received
0923 22:09:20.293698 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:20.293700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:20.293713 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:20.293745 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:20.294594 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.295313 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.296870 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.297085 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.297121 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.299340 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:20.299342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:20.299343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:20.299351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:20.301475 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:20.301985 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:20.301986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:20.301986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:20.302048 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:20.302315 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:20.302316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:20.302316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:20.302435 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:20.302511 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:20.302815 (+ 304us) server_negotiation.cc:299] Negotiation successful
0923 22:09:20.302894 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:09:20.303188 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.292608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37241 (local address 127.1.244.195:32779)
0923 22:09:20.292781 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:20.292824 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:20.293570 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:20.293947 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:20.293952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:20.294240 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:09:20.294464 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.294469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.295436 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.295437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.296738 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.296741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.297196 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.297197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.297307 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.297902 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:09:20.297910 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:09:20.299240 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:09:20.301576 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:20.301583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:20.301585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:20.301894 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:20.302116 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:20.302117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:20.302118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:20.302243 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:20.302515 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:09:20.302523 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:09:20.302671 (+ 148us) client_negotiation.cc:769] Sending connection context
0923 22:09:20.302784 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:09:20.302939 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260923 22:09:20.303807 13704 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37241
I20260923 22:09:20.303911 13704 heartbeater.cc:464] Registering TS with master...
I20260923 22:09:20.304100 13704 heartbeater.cc:511] Master 127.1.244.254:37241 requested a full tablet report, sending...
I20260923 22:09:20.304474 13220 ts_manager.cc:194] Registered new tserver with Master: 2bf82557f0294aedb57e5685008a59c7 (127.1.244.195:39115)
I20260923 22:09:20.304965 13220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:32779
I20260923 22:09:20.304992 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:09:20.315091 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.306624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.1.244.254:37241)
0923 22:09:20.307222 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:20.307224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:20.307234 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:09:20.307260 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:20.307261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:20.307271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:20.307339 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:20.308127 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.308860 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.309952 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.310262 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.310339 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.311958 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:20.311959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:20.311960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:20.311967 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:20.313718 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:20.314163 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:20.314165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:20.314165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:20.314228 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:20.314449 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:20.314450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:20.314451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:20.314585 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:20.314662 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:20.314901 (+ 239us) server_negotiation.cc:299] Negotiation successful
0923 22:09:20.314989 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563}
I20260923 22:09:20.317353 13220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:09:20.320590 13220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:09:20.332858 13717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.323579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:37029 (local address 127.0.0.1:51602)
0923 22:09:20.325628 (+ 2049us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:20.325690 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:20.325782 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:20.327971 (+ 2189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:20.327975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:20.327977 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:20.328052 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.328054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.329745 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.329746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.331133 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.331137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.332544 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.332545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.332628 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.332642 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:09:20.332718 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 22:09:20.332771 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2015,"thread_start_us":75,"threads_started":1}
I20260923 22:09:20.333580 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.323697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51602 (local address 127.1.244.193:37029)
0923 22:09:20.323855 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:20.323860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:20.325803 (+ 1943us) server_negotiation.cc:407] Connection header received
0923 22:09:20.327785 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:20.327789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:20.327809 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:20.327880 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:20.328133 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.329610 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.331466 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.332369 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.332441 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.332508 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:20.333448 (+ 940us) server_negotiation.cc:299] Negotiation successful
0923 22:09:20.333494 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":56,"threads_started":1}
I20260923 22:09:20.334528 13366 tablet_service.cc:1467] Processing CreateTablet for tablet d0d4496c4c2d46ecbbf0b819a089b231 (DEFAULT_TABLE table=test-table [id=450dfa642fbe44b6998ffb518460bc70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:09:20.334903 13366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0d4496c4c2d46ecbbf0b819a089b231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:09:20.336299 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.326031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:44093 (local address 127.0.0.1:58586)
0923 22:09:20.331811 (+ 5780us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:20.331819 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:20.331851 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:20.332282 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:20.332283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:20.332285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:20.332329 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.332330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.333673 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.333674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.334938 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.334941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.336021 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.336022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.336089 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.336095 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:09:20.336165 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:09:20.336212 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5767,"thread_start_us":64,"threads_started":1}
I20260923 22:09:20.336599 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.326268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58586 (local address 127.1.244.194:44093)
0923 22:09:20.331976 (+ 5708us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:20.331982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:20.331991 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:09:20.332031 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:20.332033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:20.332049 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:20.332201 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:20.332405 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.333573 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.335052 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.335839 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.335911 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.335984 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:20.336473 (+ 489us) server_negotiation.cc:299] Negotiation successful
0923 22:09:20.336518 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5665,"thread_start_us":88,"threads_started":1}
I20260923 22:09:20.337252 13497 tablet_service.cc:1467] Processing CreateTablet for tablet d0d4496c4c2d46ecbbf0b819a089b231 (DEFAULT_TABLE table=test-table [id=450dfa642fbe44b6998ffb518460bc70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:09:20.337586 13497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0d4496c4c2d46ecbbf0b819a089b231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:09:20.343925 13723 tablet_bootstrap.cc:492] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d: Bootstrap starting.
I20260923 22:09:20.344568 13724 tablet_bootstrap.cc:492] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: Bootstrap starting.
I20260923 22:09:20.345865 13723 tablet_bootstrap.cc:654] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d: Neither blocks nor log segments found. Creating new log.
I20260923 22:09:20.346989 13724 tablet_bootstrap.cc:654] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: Neither blocks nor log segments found. Creating new log.
I20260923 22:09:20.347402 13724 log.cc:824] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: Log is configured to *not* fsync() on all Append() calls
I20260923 22:09:20.347677 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.325826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39115 (local address 127.0.0.1:52780)
0923 22:09:20.331669 (+ 5843us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:20.331681 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:20.331734 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:20.343721 (+ 11987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:20.343722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:20.343726 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:20.343784 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.343786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.345188 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.345190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.346367 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.346370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.347402 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.347403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.347474 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.347479 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:09:20.347510 (+ 31us) client_negotiation.cc:240] Negotiation successful
0923 22:09:20.347593 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5823,"thread_start_us":81,"threads_started":1}
I20260923 22:09:20.348119 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.325926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52780 (local address 127.1.244.195:39115)
0923 22:09:20.339593 (+ 13667us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:20.339598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:20.339722 (+ 124us) server_negotiation.cc:407] Connection header received
0923 22:09:20.339767 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:20.339769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:20.339903 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:20.343706 (+ 3803us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:20.343893 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.345066 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.346500 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.347296 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.347881 (+ 585us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.347973 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:20.348001 (+ 28us) server_negotiation.cc:299] Negotiation successful
0923 22:09:20.348041 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13498,"thread_start_us":87,"threads_started":1}
I20260923 22:09:20.348464 13723 log.cc:824] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:09:20.348807 13638 tablet_service.cc:1467] Processing CreateTablet for tablet d0d4496c4c2d46ecbbf0b819a089b231 (DEFAULT_TABLE table=test-table [id=450dfa642fbe44b6998ffb518460bc70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:09:20.349179 13638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0d4496c4c2d46ecbbf0b819a089b231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:09:20.350066 13724 tablet_bootstrap.cc:492] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: No bootstrap required, opened a new log
I20260923 22:09:20.350144 13724 ts_tablet_manager.cc:1397] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260923 22:09:20.352355 13724 raft_consensus.cc:348] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } }
I20260923 22:09:20.352612 13727 tablet_bootstrap.cc:492] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: Bootstrap starting.
I20260923 22:09:20.352648 13724 raft_consensus.cc:374] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:09:20.352784 13724 raft_consensus.cc:729] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 494a20ed6a0a4869a66225ca1b6db383, State: Initialized, Role: FOLLOWER
I20260923 22:09:20.352972 13724 consensus_queue.cc:260] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } }
I20260923 22:09:20.353374 13724 ts_tablet_manager.cc:1428] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:09:20.353621 13727 tablet_bootstrap.cc:654] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: Neither blocks nor log segments found. Creating new log.
I20260923 22:09:20.353755 13569 heartbeater.cc:503] Master 127.1.244.254:37241 was elected leader, sending a full tablet report...
I20260923 22:09:20.353932 13727 log.cc:824] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: Log is configured to *not* fsync() on all Append() calls
I20260923 22:09:20.354246 13723 tablet_bootstrap.cc:492] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d: No bootstrap required, opened a new log
I20260923 22:09:20.354321 13723 ts_tablet_manager.cc:1397] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260923 22:09:20.355229 13727 tablet_bootstrap.cc:492] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: No bootstrap required, opened a new log
I20260923 22:09:20.355284 13727 ts_tablet_manager.cc:1397] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:09:20.356961 13723 raft_consensus.cc:348] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } }
I20260923 22:09:20.357125 13723 raft_consensus.cc:374] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:09:20.357169 13723 raft_consensus.cc:729] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3df6369f304744dc8fdec4be1f81783d, State: Initialized, Role: FOLLOWER
I20260923 22:09:20.357300 13723 consensus_queue.cc:260] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } }
I20260923 22:09:20.357558 13723 ts_tablet_manager.cc:1428] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:09:20.357671 13432 heartbeater.cc:503] Master 127.1.244.254:37241 was elected leader, sending a full tablet report...
I20260923 22:09:20.357610 13727 raft_consensus.cc:348] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } }
I20260923 22:09:20.357758 13727 raft_consensus.cc:374] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:09:20.357790 13727 raft_consensus.cc:729] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf82557f0294aedb57e5685008a59c7, State: Initialized, Role: FOLLOWER
I20260923 22:09:20.357890 13727 consensus_queue.cc:260] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } }
I20260923 22:09:20.358949 13727 ts_tablet_manager.cc:1428] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 22:09:20.359149 13704 heartbeater.cc:503] Master 127.1.244.254:37241 was elected leader, sending a full tablet report...
I20260923 22:09:20.374464 13730 raft_consensus.cc:482] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:09:20.374622 13730 raft_consensus.cc:504] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } }
I20260923 22:09:20.375056 13730 leader_election.cc:288] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bf82557f0294aedb57e5685008a59c7 (127.1.244.195:39115), 494a20ed6a0a4869a66225ca1b6db383 (127.1.244.194:44093)
I20260923 22:09:20.379388 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.375496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:39671 (local address 127.1.244.194:44093)
0923 22:09:20.375567 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:20.375569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:20.376096 (+ 527us) server_negotiation.cc:407] Connection header received
0923 22:09:20.376200 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:20.376201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:20.376209 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:20.376235 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:20.376427 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.377116 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.378528 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.379027 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.379072 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.379080 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:20.379281 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 22:09:20.379318 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:09:20.379514 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.375458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:38829 (local address 127.1.244.195:39115)
0923 22:09:20.375501 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:20.375503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:20.375751 (+ 248us) server_negotiation.cc:407] Connection header received
0923 22:09:20.375861 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:20.375863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:20.375874 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:20.375904 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:20.376142 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.376855 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.378552 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.379094 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.379146 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.379153 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:20.379388 (+ 235us) server_negotiation.cc:299] Negotiation successful
0923 22:09:20.379429 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:09:20.379938 13733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.375287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:44093 (local address 127.1.244.193:39671)
0923 22:09:20.376043 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:20.376053 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:20.376121 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:20.376293 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:20.376294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:20.376295 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:20.376365 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.376367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.377239 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.377240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.378424 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.378427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.379132 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.379143 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.379208 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.379211 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:09:20.379507 (+ 296us) client_negotiation.cc:240] Negotiation successful
0923 22:09:20.379582 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":48,"thread_start_us":88,"threads_started":1}
I20260923 22:09:20.379948 13732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.375232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39115 (local address 127.1.244.193:38829)
0923 22:09:20.375672 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:20.375685 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:20.375782 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:20.375972 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:20.375973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:20.375986 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:20.376067 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.376069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.376983 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.376984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.378444 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:20.378447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.379226 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.379227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:20.379303 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.379307 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:09:20.379750 (+ 443us) client_negotiation.cc:240] Negotiation successful
0923 22:09:20.379795 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":38,"thread_start_us":109,"threads_started":1}
I20260923 22:09:20.380437 13514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0d4496c4c2d46ecbbf0b819a089b231" candidate_uuid: "3df6369f304744dc8fdec4be1f81783d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "494a20ed6a0a4869a66225ca1b6db383" is_pre_election: true
I20260923 22:09:20.380501 13656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0d4496c4c2d46ecbbf0b819a089b231" candidate_uuid: "3df6369f304744dc8fdec4be1f81783d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf82557f0294aedb57e5685008a59c7" is_pre_election: true
I20260923 22:09:20.380610 13514 raft_consensus.cc:2454] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3df6369f304744dc8fdec4be1f81783d in term 0.
I20260923 22:09:20.380669 13656 raft_consensus.cc:2454] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3df6369f304744dc8fdec4be1f81783d in term 0.
I20260923 22:09:20.380865 13322 leader_election.cc:302] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3df6369f304744dc8fdec4be1f81783d, 494a20ed6a0a4869a66225ca1b6db383; no voters:
I20260923 22:09:20.380981 13730 raft_consensus.cc:2781] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:09:20.381036 13730 raft_consensus.cc:482] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:09:20.381062 13730 raft_consensus.cc:3037] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:09:20.381806 13730 raft_consensus.cc:504] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } }
I20260923 22:09:20.382079 13656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0d4496c4c2d46ecbbf0b819a089b231" candidate_uuid: "3df6369f304744dc8fdec4be1f81783d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf82557f0294aedb57e5685008a59c7"
I20260923 22:09:20.382149 13656 raft_consensus.cc:3037] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:09:20.382854 13656 raft_consensus.cc:2454] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3df6369f304744dc8fdec4be1f81783d in term 1.
I20260923 22:09:20.383036 13320 leader_election.cc:302] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bf82557f0294aedb57e5685008a59c7, 3df6369f304744dc8fdec4be1f81783d; no voters:
I20260923 22:09:20.383273 13514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0d4496c4c2d46ecbbf0b819a089b231" candidate_uuid: "3df6369f304744dc8fdec4be1f81783d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "494a20ed6a0a4869a66225ca1b6db383"
I20260923 22:09:20.383380 13514 raft_consensus.cc:3037] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:09:20.384253 13514 raft_consensus.cc:2454] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3df6369f304744dc8fdec4be1f81783d in term 1.
I20260923 22:09:20.384444 13730 leader_election.cc:288] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [CANDIDATE]: Term 1 election: Requested vote from peers 2bf82557f0294aedb57e5685008a59c7 (127.1.244.195:39115), 494a20ed6a0a4869a66225ca1b6db383 (127.1.244.194:44093)
I20260923 22:09:20.384513 13730 raft_consensus.cc:2781] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:09:20.384557 13730 raft_consensus.cc:686] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [term 1 LEADER]: Becoming Leader. State: Replica: 3df6369f304744dc8fdec4be1f81783d, State: Running, Role: LEADER
I20260923 22:09:20.384645 13730 consensus_queue.cc:237] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } }
I20260923 22:09:20.385452 13220 catalog_manager.cc:5495] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3df6369f304744dc8fdec4be1f81783d (127.1.244.193). New cstate: current_term: 1 leader_uuid: "3df6369f304744dc8fdec4be1f81783d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3df6369f304744dc8fdec4be1f81783d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 } health_report { overall_health: UNKNOWN } } }
I20260923 22:09:20.394184 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.390709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.1.244.193:37029)
0923 22:09:20.390753 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:20.390755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:20.390764 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:09:20.390848 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:20.390849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:20.390868 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:20.390967 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:09:20.391110 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.391901 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.393064 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:20.393383 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:20.393443 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:20.393637 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:09:20.393773 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:09:20.393844 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:20.394053 (+ 209us) server_negotiation.cc:299] Negotiation successful
0923 22:09:20.394073 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:09:20.396850 13514 raft_consensus.cc:1261] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [term 1 FOLLOWER]: Refusing update from remote peer 3df6369f304744dc8fdec4be1f81783d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:09:20.396873 13656 raft_consensus.cc:1261] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [term 1 FOLLOWER]: Refusing update from remote peer 3df6369f304744dc8fdec4be1f81783d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:09:20.397279 13730 consensus_queue.cc:1035] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bf82557f0294aedb57e5685008a59c7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:09:20.397452 13730 consensus_queue.cc:1035] T d0d4496c4c2d46ecbbf0b819a089b231 P 3df6369f304744dc8fdec4be1f81783d [LEADER]: Connected to new peer: Peer: permanent_uuid: "494a20ed6a0a4869a66225ca1b6db383" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:09:20.401384 13737 mvcc.cc:205] Tried to move back new op lower bound from 7332664772182646784 to 7332664772136329216. Current Snapshot: MvccSnapshot[applied={T|T < 7332664772182646784 or (T in {7332664772182646784})}]
I20260923 22:09:20.403474 13740 mvcc.cc:205] Tried to move back new op lower bound from 7332664772182646784 to 7332664772136329216. Current Snapshot: MvccSnapshot[applied={T|T < 7332664772182646784}]
I20260923 22:09:20.404146 13219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:09:20.404269 13219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:09:20.404989 13219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ef97230077549caadf0ea0e9ac299f7: Sending DeleteTablet for 3 replicas of tablet d0d4496c4c2d46ecbbf0b819a089b231
I20260923 22:09:20.405416 13366 tablet_service.cc:1514] Processing DeleteTablet for tablet d0d4496c4c2d46ecbbf0b819a089b231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:09:20 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:51602
I20260923 22:09:20.405615 13638 tablet_service.cc:1514] Processing DeleteTablet for tablet d0d4496c4c2d46ecbbf0b819a089b231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:09:20 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:52780
I20260923 22:09:20.405817 13748 tablet_replica.cc:330] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: stopping tablet replica
I20260923 22:09:20.405901 13497 tablet_service.cc:1514] Processing DeleteTablet for tablet d0d4496c4c2d46ecbbf0b819a089b231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:09:20 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:58586
I20260923 22:09:20.405952 13748 raft_consensus.cc:2229] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:09:20.406157 13748 raft_consensus.cc:2258] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:09:20.406307 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 13301
Sep 23 22:09:20 dist-test-slave-b5jz krb5kdc[12744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:09:20.408819 13748 ts_tablet_manager.cc:1905] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:09:20.410248 13749 tablet_replica.cc:330] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: stopping tablet replica
I20260923 22:09:20.410401 13749 raft_consensus.cc:2229] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:09:20.410478 13749 raft_consensus.cc:2258] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:09:20.410772 13748 ts_tablet_manager.cc:1918] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:09:20.410892 13748 log.cc:1196] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/d0d4496c4c2d46ecbbf0b819a089b231
I20260923 22:09:20.411265 13748 ts_tablet_manager.cc:1939] T d0d4496c4c2d46ecbbf0b819a089b231 P 2bf82557f0294aedb57e5685008a59c7: Deleting consensus metadata
I20260923 22:09:20.411798 13205 catalog_manager.cc:4841] TS 2bf82557f0294aedb57e5685008a59c7 (127.1.244.195:39115): tablet d0d4496c4c2d46ecbbf0b819a089b231 (table test-table [id=450dfa642fbe44b6998ffb518460bc70]) successfully deleted
I20260923 22:09:20.412002 13749 ts_tablet_manager.cc:1905] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:09:20.413607 13749 ts_tablet_manager.cc:1918] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:09:20.413692 13749 log.cc:1196] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/d0d4496c4c2d46ecbbf0b819a089b231
I20260923 22:09:20.414294 13749 ts_tablet_manager.cc:1939] T d0d4496c4c2d46ecbbf0b819a089b231 P 494a20ed6a0a4869a66225ca1b6db383: Deleting consensus metadata
I20260923 22:09:20.414739 13207 catalog_manager.cc:4841] TS 494a20ed6a0a4869a66225ca1b6db383 (127.1.244.194:44093): tablet d0d4496c4c2d46ecbbf0b819a089b231 (table test-table [id=450dfa642fbe44b6998ffb518460bc70]) successfully deleted
W20260923 22:09:20.418159 13206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:37029 (error 108)
I20260923 22:09:20.418706 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:20.418559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:37029 (local address 127.0.0.1:51618)
0923 22:09:20.418623 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:20.418670 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:37029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 22:09:20.418697 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 13438
W20260923 22:09:20.421242 13206 catalog_manager.cc:4584] TS 3df6369f304744dc8fdec4be1f81783d (127.1.244.193:37029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0d4496c4c2d46ecbbf0b819a089b231: Network error: Client connection negotiation failed: client connection to 127.1.244.193:37029: connect: Connection refused (error 111)
I20260923 22:09:20.427937 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 13575
I20260923 22:09:20.433745 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 13188
I20260923 22:09:20.440397 2003 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 22:09:20.795608 2003 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 22:09:20.795704 2003 mini_ranger.cc:66] Stopping Ranger...
I20260923 22:09:21.140308 2003 mini_ranger.cc:68] Stopped Ranger
2026-09-23 22:09:21.140 UTC [12774] LOG: received smart shutdown request
2026-09-23 22:09:21.144 UTC [12774] LOG: background worker "logical replication launcher" (PID 12783) exited with exit code 1
2026-09-23 22:09:21.145 UTC [12778] LOG: shutting down
2026-09-23 22:09:21.317 UTC [12774] LOG: database system is shut down
2026-09-23T22:09:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45034 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:09:21 dist-test-slave-b5jz krb5kdc[13759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:09:21 dist-test-slave-b5jz krb5kdc[13759](info): set up 2 sockets
Sep 23 22:09:21 dist-test-slave-b5jz krb5kdc[13759](info): commencing operation
krb5kdc: starting...
W20260923 22:09:22.459508 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:09:22 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:09:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:09:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/rangeradmin_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/rangeradmin_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/rangerlookup_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/rangerlookup_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
I20260923 22:09:22.624366 2003 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6wdnYN/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/postgres -l logfile start
2026-09-23 22:09:25.019 UTC [13789] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 22:09:25.019 UTC [13789] LOG: listening on IPv4 address "127.1.244.212", port 34375
2026-09-23 22:09:25.023 UTC [13789] LOG: listening on Unix socket "/tmp/.s.PGSQL.34375"
2026-09-23 22:09:25.057 UTC [13792] LOG: database system was shut down at 2026-09-23 22:09:23 UTC
2026-09-23 22:09:25.061 UTC [13789] LOG: database system is ready to accept connections
I20260923 22:09:26.007244 2003 mini_postgres.cc:96] Postgres bound to 34375
2026-09-23 22:09:26.013 UTC [13800] FATAL: database "slave" does not exist
127.1.244.212:34375 - accepting connections
I20260923 22:09:26.013962 2003 mini_ranger.cc:161] Starting Ranger...
I20260923 22:09:26.028249 2003 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 22:09:26.466343 2003 mini_ranger.cc:87] Created ranger Postgres database
I20260923 22:09:26.466427 2003 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-admin
2026-09-23 22:09:26,726 [I] DB FLAVOR :POSTGRES
2026-09-23 22:09:26,727 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:09:26,727 [I] Checking connection..
2026-09-23 22:09:26,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 22:09:26,971 [I] Checking connection passed.
2026-09-23 22:09:26,971 [I] --------- Verifying version history table ---------
2026-09-23 22:09:26,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:09:27,229 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:09:27,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:09:27,471 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:09:27,471 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 22:09:27,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 22:09:27.718 UTC [13864] WARNING: there is no transaction in progress
2026-09-23 22:09:27,723 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 22:09:27,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:09:27,961 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 22:09:27,961 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:09:27,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 22:09:28,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 22:09:28,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-b5jz.c.gcp-upstream.internal','N') ;"
2026-09-23 22:09:28,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:09:28,939 [I] Table x_portal_user does not exist in database ranger
2026-09-23 22:09:28,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:09:29,184 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 22:09:29,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:09:29,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:09:29,647 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 22:09:29,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 22:09:30.579 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.594 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.611 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.626 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.641 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.659 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.722 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.729 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.742 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.752 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.762 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.775 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.784 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.790 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.796 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:30.803 UTC [13972] WARNING: there is no transaction in progress
2026-09-23 22:09:31,398 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 22:09:31,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 22:09:31,632 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 22:09:31,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:09:31,878 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 22:09:31,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:09:32,106 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 22:09:32,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:09:32,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:09:32,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-b5jz.c.gcp-upstream.internal';"
2026-09-23 22:09:32,828 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 22:09:32,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 22:09:33,071 [I] DB_PATCHES have already been applied
I20260923 22:09:33.077199 2003 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-admin:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6wdnYN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 22:09:33.077260 2003 mini_ranger.cc:193] Using host: 127.1.244.212
I20260923 22:09:33.080235 2003 mini_ranger.cc:227] Ranger admin URL: http://127.1.244.212:57981
Sep 23, 2026 10:09:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:09:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:09:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 10:09:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 10:09:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 10:09:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 10:09:33 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 22:09:34.184896 2003 mini_ranger.cc:160] Time spent starting Ranger: real 8.171s user 0.000s sys 0.008s
Sep 23, 2026 10:09:54 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 10:09:54 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 23, 2026 10:09:54 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 10:09:54 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 10:09:55 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 22:09:55.569237 2003 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/rangerkms_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/rangerkms_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.244.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.244.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 22:09:55 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201395, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 22:09:55.646366 2003 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 22:09:55.654899 2003 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 22:09:56.385833 2003 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 22:09:56.385898 2003 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-kms
I20260923 22:09:56.385910 2003 mini_ranger_kms.cc:227] Using postgres at 127.1.244.212:34375
2026-09-23 22:09:56,545 [I] DB FLAVOR :POSTGRES
2026-09-23 22:09:56,545 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:09:56,545 [I] Checking connection
2026-09-23 22:09:56,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 22:09:56,778 [I] connection success
2026-09-23 22:09:56,778 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 22:09:56,778 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 22:09:56,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 22:09:57,015 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 22:09:57,015 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:09:57,016 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 22:09:57,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:34375/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 22:09:57,295 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 22:09:57.763932 2003 mini_ranger_kms.cc:310] Created kms service
I20260923 22:09:57.951058 2003 mini_ranger_kms.cc:326] Created kudu user
I20260923 22:09:57.993402 2003 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 22:09:57 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201395, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:09:58.352738 2003 mini_ranger_kms.cc:384] Added ranger policy
I20260923 22:09:58.352860 2003 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-kms:/tmp/dist-test-task6wdnYN/build/release/bin/postgresql.jar:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6wdnYN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 22:09:58.352896 2003 mini_ranger_kms.cc:240] Using host: 127.1.244.212
I20260923 22:09:58.356199 2003 mini_ranger_kms.cc:276] Ranger KMS PID: 14152
I20260923 22:09:58.356280 2003 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.244.212:41023
Sep 23, 2026 10:09:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:09:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:09:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 10:09:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 10:09:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 10:09:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 22:10:00.606010 2003 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.960s user 0.003s sys 0.139s
I20260923 22:10:00.606102 2003 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 22:10:00.606148 2003 mini_ranger_kms.cc:401] 127.1.244.212:41023/kms/v1/keys
Sep 23 22:10:00 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 22:10:01 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Sep 23 22:10:01 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Sep 23 22:10:03 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:10:03.783490 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41929
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:41929
--ranger_config_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:41023/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:03.864748 14203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:03.864951 14203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:03.864972 14203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:03.866138 14203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:03.866184 14203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:03.866199 14203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:03.866211 14203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:03.866223 14203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:03.867887 14203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.244.212:41023/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:41929
--ranger_config_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:03.868134 14203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:03.868384 14203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:03.874047 14208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:03.874037 14211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:03.874066 14209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:03.874418 14203 server_base.cc:1034] running on GCE node
I20260923 22:10:03.874670 14203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:03.874946 14203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:03.876106 14203 hybrid_clock.cc:648] HybridClock initialized: now 1790201403876080 us; error 41 us; skew 500 ppm
Sep 23 22:10:03 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:03.878855 14203 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:10:03.879357 14203 webserver.cc:460] Webserver started at http://127.1.244.254:35051/ using document root <none> and password file <none>
I20260923 22:10:03.879664 14203 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:03.879724 14203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:03.879863 14203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:10:03 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:10:03.939062 14203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "ec885e2a18ae4095926d44bf529ba131"
format_stamp: "Formatted at 2026-09-23 22:10:03 on dist-test-slave-b5jz"
server_key: "a3c29098cc40e9bbc2c7fe4890e9d78d"
server_key_iv: "39728bb768d490d40c5d62fd280350a1"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:03.939476 14203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "ec885e2a18ae4095926d44bf529ba131"
format_stamp: "Formatted at 2026-09-23 22:10:03 on dist-test-slave-b5jz"
server_key: "a3c29098cc40e9bbc2c7fe4890e9d78d"
server_key_iv: "39728bb768d490d40c5d62fd280350a1"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:03.940924 14203 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:03.956292 14218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:03.956547 14203 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:03.956674 14203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "ec885e2a18ae4095926d44bf529ba131"
format_stamp: "Formatted at 2026-09-23 22:10:03 on dist-test-slave-b5jz"
server_key: "a3c29098cc40e9bbc2c7fe4890e9d78d"
server_key_iv: "39728bb768d490d40c5d62fd280350a1"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:03.956777 14203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:04.000998 14203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:04.003392 14203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:04.003582 14203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:04.008919 14203 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:41929
I20260923 22:10:04.008966 14270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:41929 every 8 connection(s)
I20260923 22:10:04.009430 14203 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:04.009594 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 14203
I20260923 22:10:04.009716 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:10:04.009929 2003 external_mini_cluster.cc:1428] Setting key 89e8bab2e66ac391e8edd462bac3fda7
I20260923 22:10:04.010375 14271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:04.014094 14271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Bootstrap starting.
I20260923 22:10:04.014922 14271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:04.015241 14271 log.cc:824] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:04.016142 14271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: No bootstrap required, opened a new log
Sep 23 22:10:04 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:04.018232 14271 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } }
I20260923 22:10:04.018424 14271 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:04.018496 14271 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec885e2a18ae4095926d44bf529ba131, State: Initialized, Role: FOLLOWER
I20260923 22:10:04.018682 14271 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } }
I20260923 22:10:04.018771 14271 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:04.018805 14271 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:04.018872 14271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:04.019610 14271 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } }
I20260923 22:10:04.019730 14271 leader_election.cc:302] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec885e2a18ae4095926d44bf529ba131; no voters:
I20260923 22:10:04.019942 14271 leader_election.cc:288] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:10:04.019981 14276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:04.020241 14276 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 1 LEADER]: Becoming Leader. State: Replica: ec885e2a18ae4095926d44bf529ba131, State: Running, Role: LEADER
I20260923 22:10:04.020301 14271 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:04.020411 14276 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } }
I20260923 22:10:04.020776 14276 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec885e2a18ae4095926d44bf529ba131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } } }
I20260923 22:10:04.020931 14276 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:04.021119 14276 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec885e2a18ae4095926d44bf529ba131. Latest consensus state: current_term: 1 leader_uuid: "ec885e2a18ae4095926d44bf529ba131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } } }
I20260923 22:10:04.021245 14276 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:04.021420 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:04.010435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50040 (local address 127.1.244.254:41929)
0923 22:10:04.010606 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:04.010616 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:04.010720 (+ 104us) server_negotiation.cc:407] Connection header received
0923 22:10:04.011199 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:04.011207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:04.011265 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:04.011420 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:04.012131 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:04.013081 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:04.013838 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:04.014235 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:04.014348 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:04.016944 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:04.016948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:04.016949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:04.016961 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:04.019202 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:04.019580 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:04.019583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:04.019584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:04.019699 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:04.019902 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:04.019903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:04.019903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:04.020248 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:04.020463 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:04.021030 (+ 567us) server_negotiation.cc:299] Negotiation successful
0923 22:10:04.021164 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260923 22:10:04.021960 14271 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 22:10:04.021405 14283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:04.022805 14283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:04.028038 14283 catalog_manager.cc:1347] Generated new cluster ID: 7c12f74790a04c60b656a789e2598fd2
I20260923 22:10:04.028295 14283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:04.046037 14283 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:04.046948 14283 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:04.057389 14283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Generated new TSK 0
I20260923 22:10:04.057685 14283 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:10:04.779129 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:41023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41929
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:04.883467 14316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:04.883705 14316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:04.883731 14316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:04.884878 14316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:04.884919 14316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:04.884944 14316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:10:04.886435 14316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.244.212:41023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:04.886680 14316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:04.886901 14316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:04.892038 14324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:04.892294 14321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:04.892402 14322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:04.892624 14316 server_base.cc:1034] running on GCE node
I20260923 22:10:04.892815 14316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:04.893075 14316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:04.896306 14316 hybrid_clock.cc:648] HybridClock initialized: now 1790201404896282 us; error 36 us; skew 500 ppm
Sep 23 22:10:04 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:04.900089 14316 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:10:04.900694 14316 webserver.cc:460] Webserver started at http://127.1.244.193:44037/ using document root <none> and password file <none>
I20260923 22:10:04.900946 14316 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:04.901051 14316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:04.901194 14316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:10:04 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:10:04.916486 14316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "b4f932933fba41fea005ed4357383091"
format_stamp: "Formatted at 2026-09-23 22:10:04 on dist-test-slave-b5jz"
server_key: "bed12485f12f7c1bc9f0eef36d2f451f"
server_key_iv: "23f3cdeee88b38f6bb3243bc102ae3ed"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:04.917069 14316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "b4f932933fba41fea005ed4357383091"
format_stamp: "Formatted at 2026-09-23 22:10:04 on dist-test-slave-b5jz"
server_key: "bed12485f12f7c1bc9f0eef36d2f451f"
server_key_iv: "23f3cdeee88b38f6bb3243bc102ae3ed"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:04.919951 14316 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260923 22:10:04.933066 14333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:04.933290 14316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:04.933382 14316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "b4f932933fba41fea005ed4357383091"
format_stamp: "Formatted at 2026-09-23 22:10:04 on dist-test-slave-b5jz"
server_key: "bed12485f12f7c1bc9f0eef36d2f451f"
server_key_iv: "23f3cdeee88b38f6bb3243bc102ae3ed"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:04.933465 14316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:04.948201 14316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:04.950583 14316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:04.950793 14316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:04.951038 14316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:04.951447 14316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:04.951502 14316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:04.951538 14316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:04.951639 14316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:04.958166 14316 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:46415
I20260923 22:10:04.958652 14316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:04.967073 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 14316
I20260923 22:10:04.967187 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:10:04.967401 2003 external_mini_cluster.cc:1428] Setting key 94fb0eafdb055631e3dac4d947056f35
I20260923 22:10:04.971411 14446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:46415 every 8 connection(s)
Sep 23 22:10:04 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260923 22:10:04.982991 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:04.972014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:44775 (local address 127.1.244.254:41929)
0923 22:10:04.973015 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:04.973017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:04.973028 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:10:04.973056 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:04.973056 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:04.973069 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:04.973159 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:04.973982 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:04.974775 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:04.976299 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:04.976590 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:04.976673 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:04.979317 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:04.979319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:04.979319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:04.979326 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:04.981192 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:04.981726 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:04.981727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:04.981728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:04.981794 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:04.982107 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:04.982108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:04.982108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:04.982266 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:04.982348 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:04.982786 (+ 438us) server_negotiation.cc:299] Negotiation successful
0923 22:10:04.982878 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":82,"threads_started":1}
I20260923 22:10:04.983192 14450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:04.971746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41929 (local address 127.1.244.193:44775)
0923 22:10:04.972169 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:04.972219 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:04.972857 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:04.973300 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:04.973305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:04.973559 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:04.973860 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:04.973869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:04.974891 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:04.974892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:04.976198 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:04.976201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:04.976757 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:04.976758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:04.976892 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:04.977513 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:04.977524 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:04.979205 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:04.981290 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:04.981297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:04.981300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:04.981607 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:04.981880 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:04.981881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:04.981881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:04.982014 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:04.982355 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:04.982361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:04.982557 (+ 196us) client_negotiation.cc:769] Sending connection context
0923 22:10:04.982746 (+ 189us) client_negotiation.cc:240] Negotiation successful
0923 22:10:04.982913 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
I20260923 22:10:04.984014 14447 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41929
I20260923 22:10:04.984112 14447 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:04.984298 14447 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
I20260923 22:10:04.985745 14235 ts_manager.cc:194] Registered new tserver with Master: b4f932933fba41fea005ed4357383091 (127.1.244.193:46415)
I20260923 22:10:04.986608 14235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:44775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:10:05.014453 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:41023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41929
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 22:10:05.146508 14453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:05.146935 14453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:05.147013 14453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:05.148813 14453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:05.149036 14453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:05.149128 14453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:10:05.151305 14453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.244.212:41023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:05.151840 14453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:05.152163 14453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:05.158653 14461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:05.159054 14460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:05.159284 14463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:05.160334 14453 server_base.cc:1034] running on GCE node
I20260923 22:10:05.160640 14453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:05.160910 14453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:05.162065 14453 hybrid_clock.cc:648] HybridClock initialized: now 1790201405162043 us; error 37 us; skew 500 ppm
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:05.165419 14453 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:10:05.165928 14453 webserver.cc:460] Webserver started at http://127.1.244.194:40017/ using document root <none> and password file <none>
I20260923 22:10:05.166170 14453 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:05.166219 14453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:05.166342 14453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:10:05.182818 14453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "ed524fe4d67544129dfd95bd6bb03af9"
format_stamp: "Formatted at 2026-09-23 22:10:05 on dist-test-slave-b5jz"
server_key: "d90fd62910a25063d5ea361788e85e3f"
server_key_iv: "07055a6dc23e1943eb35776314de3ec8"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:05.183357 14453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "ed524fe4d67544129dfd95bd6bb03af9"
format_stamp: "Formatted at 2026-09-23 22:10:05 on dist-test-slave-b5jz"
server_key: "d90fd62910a25063d5ea361788e85e3f"
server_key_iv: "07055a6dc23e1943eb35776314de3ec8"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:05.186456 14453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.001s
I20260923 22:10:05.205718 14470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.205950 14453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260923 22:10:05.206029 14453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "ed524fe4d67544129dfd95bd6bb03af9"
format_stamp: "Formatted at 2026-09-23 22:10:05 on dist-test-slave-b5jz"
server_key: "d90fd62910a25063d5ea361788e85e3f"
server_key_iv: "07055a6dc23e1943eb35776314de3ec8"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:05.206108 14453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:05.246222 14453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:05.248656 14453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:05.248853 14453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:05.249094 14453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:05.249475 14453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:05.249558 14453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.249626 14453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:05.249694 14453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.257876 14453 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:39531
I20260923 22:10:05.258356 14453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:05.259029 14584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:39531 every 8 connection(s)
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:05.266836 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 14453
I20260923 22:10:05.266939 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:10:05.267129 2003 external_mini_cluster.cc:1428] Setting key f325fc033a887a49ffc01c3da2c27415
I20260923 22:10:05.271041 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:05.259708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:41863 (local address 127.1.244.254:41929)
0923 22:10:05.259758 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:05.259760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:05.260332 (+ 572us) server_negotiation.cc:407] Connection header received
0923 22:10:05.261063 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:05.261065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:05.261078 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:05.261108 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:05.262004 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.262747 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.264235 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.264543 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.264589 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:05.267213 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:05.267214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:05.267214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:05.267222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:05.269494 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:05.270003 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:05.270005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:05.270005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:05.270064 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:05.270333 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:05.270334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:05.270334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:05.270461 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:05.270501 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:05.270861 (+ 360us) server_negotiation.cc:299] Negotiation successful
0923 22:10:05.270948 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:10:05.271806 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:05.259862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41929 (local address 127.1.244.194:41863)
0923 22:10:05.260205 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:05.260246 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:05.260940 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:05.261332 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:05.261337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:05.261565 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:05.261851 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:05.261907 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.262886 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.262887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:05.264133 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:05.264136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.264700 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.264701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:05.264839 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:05.265420 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:05.265430 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:05.267105 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:05.269597 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:05.269604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:05.269606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:05.269914 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:05.270133 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:05.270133 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:05.270134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:05.270263 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:05.270542 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:05.270547 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:05.270717 (+ 170us) client_negotiation.cc:769] Sending connection context
0923 22:10:05.271373 (+ 656us) client_negotiation.cc:240] Negotiation successful
0923 22:10:05.271531 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":79,"threads_started":1}
I20260923 22:10:05.272426 14585 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41929
I20260923 22:10:05.272511 14585 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:05.272696 14585 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
I20260923 22:10:05.273073 14235 ts_manager.cc:194] Registered new tserver with Master: ed524fe4d67544129dfd95bd6bb03af9 (127.1.244.194:39531)
I20260923 22:10:05.273532 14235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:41863
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:10:05.304194 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:41023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41929
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:05.422971 14590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:05.423318 14590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:05.423401 14590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:05.425015 14590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:05.425123 14590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:05.425200 14590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:10:05.427217 14590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.244.212:41023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:05.427697 14590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:05.427994 14590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:05.434463 14598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:05.435688 14590 server_base.cc:1034] running on GCE node
W20260923 22:10:05.435796 14595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:05.435703 14596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:05.436192 14590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:05.436440 14590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:05.437604 14590 hybrid_clock.cc:648] HybridClock initialized: now 1790201405437583 us; error 35 us; skew 500 ppm
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:05.441089 14590 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:10:05.441555 14590 webserver.cc:460] Webserver started at http://127.1.244.195:37923/ using document root <none> and password file <none>
I20260923 22:10:05.441802 14590 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:05.441847 14590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:05.441962 14590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:10:05.458873 14590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "252330e68d594f1d9704b637466cb393"
format_stamp: "Formatted at 2026-09-23 22:10:05 on dist-test-slave-b5jz"
server_key: "2824cc896ef799626fd56559f317db5e"
server_key_iv: "85b53e83153259b30971a08bcb9285a5"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:05.459406 14590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "252330e68d594f1d9704b637466cb393"
format_stamp: "Formatted at 2026-09-23 22:10:05 on dist-test-slave-b5jz"
server_key: "2824cc896ef799626fd56559f317db5e"
server_key_iv: "85b53e83153259b30971a08bcb9285a5"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:05.462024 14590 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:05.478407 14605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.478696 14590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:05.478844 14590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "252330e68d594f1d9704b637466cb393"
format_stamp: "Formatted at 2026-09-23 22:10:05 on dist-test-slave-b5jz"
server_key: "2824cc896ef799626fd56559f317db5e"
server_key_iv: "85b53e83153259b30971a08bcb9285a5"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:05.478941 14590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:05.498705 14590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:05.501330 14590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:05.501540 14590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:05.501821 14590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:05.502185 14590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:05.502235 14590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.502267 14590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:05.502326 14590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.512377 14590 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:42157
I20260923 22:10:05.512897 14590 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:05.516307 14718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:42157 every 8 connection(s)
I20260923 22:10:05.521059 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 14590
I20260923 22:10:05.521162 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:10:05.521353 2003 external_mini_cluster.cc:1428] Setting key 020ee6a344ddb34845ff4f73d93df174
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:05.527438 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:05.515218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:60501 (local address 127.1.244.254:41929)
0923 22:10:05.515264 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:05.515266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:05.515530 (+ 264us) server_negotiation.cc:407] Connection header received
0923 22:10:05.516359 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:05.516360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:05.516375 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:05.516446 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:05.517227 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.518022 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.519468 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.519829 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.519899 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:05.524080 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:05.524082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:05.524082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:05.524090 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:05.525838 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:05.526335 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:05.526336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:05.526337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:05.526396 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:05.526662 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:05.526663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:05.526663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:05.526807 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:05.526848 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:05.527256 (+ 408us) server_negotiation.cc:299] Negotiation successful
0923 22:10:05.527347 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:10:05.527688 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:05.515203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41929 (local address 127.1.244.195:60501)
0923 22:10:05.515367 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:05.515418 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:05.516213 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:05.516586 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:05.516591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:05.516831 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:05.517119 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:05.517125 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.518136 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.518137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:05.519367 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:05.519370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.519992 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.519994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:05.520129 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:05.520788 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:05.520799 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:05.523968 (+ 3169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:05.525939 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:05.525946 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:05.525948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:05.526241 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:05.526466 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:05.526467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:05.526467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:05.526593 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:05.526896 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:05.526901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:05.527069 (+ 168us) client_negotiation.cc:769] Sending connection context
0923 22:10:05.527205 (+ 136us) client_negotiation.cc:240] Negotiation successful
0923 22:10:05.527359 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260923 22:10:05.528195 14719 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41929
I20260923 22:10:05.528322 14719 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:05.528532 14719 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
I20260923 22:10:05.528904 14235 ts_manager.cc:194] Registered new tserver with Master: 252330e68d594f1d9704b637466cb393 (127.1.244.195:42157)
I20260923 22:10:05.529299 14235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:60501
I20260923 22:10:05.532917 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:05.533489 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 14316
I20260923 22:10:05.539705 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 14453
I20260923 22:10:05.546252 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 14590
I20260923 22:10:05.551384 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 14203
I20260923 22:10:05.556422 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41929
--webserver_interface=127.1.244.254
--webserver_port=35051
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:41929
--ranger_config_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:41023/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:41023/kms with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:05.640488 14722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:05.640695 14722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:05.640718 14722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:05.641820 14722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:05.641857 14722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:05.641872 14722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:05.641885 14722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:05.641897 14722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:05.643339 14722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:41023/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:41929
--ranger_config_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=35051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:05.643510 14722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:05.643731 14722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:05.648310 14730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:05.648315 14728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:05.648326 14727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:05.648625 14722 server_base.cc:1034] running on GCE node
I20260923 22:10:05.648916 14722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:05.649204 14722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:05.650380 14722 hybrid_clock.cc:648] HybridClock initialized: now 1790201405650372 us; error 38 us; skew 500 ppm
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:05.652942 14722 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:10:05.653326 14722 webserver.cc:460] Webserver started at http://127.1.244.254:35051/ using document root <none> and password file <none>
I20260923 22:10:05.653501 14722 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:05.653548 14722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:10:05.702740 14722 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:05.703675 14738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.703915 14722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:05.704042 14722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "ec885e2a18ae4095926d44bf529ba131"
format_stamp: "Formatted at 2026-09-23 22:10:03 on dist-test-slave-b5jz"
server_key: "a3c29098cc40e9bbc2c7fe4890e9d78d"
server_key_iv: "39728bb768d490d40c5d62fd280350a1"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:05.704543 14722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:05.718034 14722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:05.719771 14722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:05.719900 14722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:05.724346 14722 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:41929
I20260923 22:10:05.724390 14790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:41929 every 8 connection(s)
I20260923 22:10:05.724767 14722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:05.725904 14791 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:10:05.726552 14791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Bootstrap starting.
I20260923 22:10:05.732545 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 14722
I20260923 22:10:05.732903 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:41023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:46415
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:41929
--webserver_port=44037
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:41023/kms with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:05.735163 14791 log.cc:824] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:05.737090 14791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:05.737390 14791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Bootstrap complete.
I20260923 22:10:05.740137 14791 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } }
I20260923 22:10:05.740406 14791 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec885e2a18ae4095926d44bf529ba131, State: Initialized, Role: FOLLOWER
I20260923 22:10:05.740589 14791 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } }
I20260923 22:10:05.740669 14791 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:05.740727 14791 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:05.740780 14791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:10:05.741900 14791 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } }
I20260923 22:10:05.741995 14791 leader_election.cc:302] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec885e2a18ae4095926d44bf529ba131; no voters:
I20260923 22:10:05.742226 14791 leader_election.cc:288] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:10:05.742267 14795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:10:05.742393 14795 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [term 2 LEADER]: Becoming Leader. State: Replica: ec885e2a18ae4095926d44bf529ba131, State: Running, Role: LEADER
I20260923 22:10:05.742524 14791 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:05.742595 14795 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } }
I20260923 22:10:05.742995 14797 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec885e2a18ae4095926d44bf529ba131. Latest consensus state: current_term: 2 leader_uuid: "ec885e2a18ae4095926d44bf529ba131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } } }
I20260923 22:10:05.743095 14797 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:05.743345 14796 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ec885e2a18ae4095926d44bf529ba131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec885e2a18ae4095926d44bf529ba131" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41929 } } }
I20260923 22:10:05.743462 14802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:05.743481 14796 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:05.744096 14802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:05.744237 14802 catalog_manager.cc:1259] Loaded cluster ID: 7c12f74790a04c60b656a789e2598fd2
I20260923 22:10:05.744310 14802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:05.744553 14791 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 22:10:05.747844 14802 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:05.748473 14802 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Loaded TSK: 0
I20260923 22:10:05.748796 14802 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:10:05.843683 14793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:05.843950 14793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:05.843987 14793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:05.845803 14793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:05.845937 14793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:05.846019 14793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:10:05.848276 14793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:41023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:46415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=44037
--tserver_master_addrs=127.1.244.254:41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:05.848515 14793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:05.848786 14793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:05.854849 14829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:05.855465 14831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:05.858649 14793 server_base.cc:1034] running on GCE node
W20260923 22:10:05.859210 14828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:05.859458 14793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:05.859791 14793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:05.860960 14793 hybrid_clock.cc:648] HybridClock initialized: now 1790201405860937 us; error 54 us; skew 500 ppm
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:05.864203 14793 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:10:05.864738 14793 webserver.cc:460] Webserver started at http://127.1.244.193:44037/ using document root <none> and password file <none>
I20260923 22:10:05.865001 14793 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:05.865087 14793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:10:05.888007 14793 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260923 22:10:05.891074 14839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.891263 14793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260923 22:10:05.891355 14793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "b4f932933fba41fea005ed4357383091"
format_stamp: "Formatted at 2026-09-23 22:10:04 on dist-test-slave-b5jz"
server_key: "bed12485f12f7c1bc9f0eef36d2f451f"
server_key_iv: "23f3cdeee88b38f6bb3243bc102ae3ed"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:05.892482 14793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:05.918431 14793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:05.920765 14793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:05.920954 14793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:05.921231 14793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:05.921643 14793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:05.921693 14793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.921723 14793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:05.921742 14793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:05.929170 14793 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:46415
I20260923 22:10:05.929267 14952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:46415 every 8 connection(s)
I20260923 22:10:05.929646 14793 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:05.930317 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 14793
I20260923 22:10:05.930610 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:41023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:39531
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:41929
--webserver_port=40017
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:41023/kms with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:10:05 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:05.946805 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:05.930407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41929 (local address 127.1.244.193:53305)
0923 22:10:05.935218 (+ 4811us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:05.935264 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:05.935883 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:05.936456 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:05.936461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:05.936661 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:05.936849 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:05.936853 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.938150 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.938151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:05.939089 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:05.939092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.940078 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.940079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:05.940229 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:05.940841 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:05.940852 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:05.942459 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:05.944907 (+ 2448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:05.944913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:05.944915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:05.945196 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:05.945486 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:05.945487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:05.945487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:05.945605 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:05.946130 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:05.946136 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:05.946301 (+ 165us) client_negotiation.cc:769] Sending connection context
0923 22:10:05.946414 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:10:05.946571 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4765,"thread_start_us":81,"threads_started":1}
I20260923 22:10:05.947649 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:05.930482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53305 (local address 127.1.244.254:41929)
0923 22:10:05.936077 (+ 5595us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:05.936085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:05.936113 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:10:05.936228 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:05.936233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:05.936278 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:05.936419 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:05.936972 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.938042 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.939191 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:05.939858 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:05.939967 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:05.942608 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:05.942617 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:05.942618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:05.942628 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:05.944812 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:05.945306 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:05.945310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:05.945310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:05.945416 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:05.945672 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:05.945673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:05.945674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:05.946016 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:05.946100 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:05.947296 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0923 22:10:05.947417 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5539,"thread_start_us":73,"threads_started":1}
I20260923 22:10:05.948688 14953 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41929
I20260923 22:10:05.948796 14953 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:05.948992 14953 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
W20260923 22:10:05.949904 14953 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:41929 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:10:05.949983 14953 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:05.950048 14953 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
I20260923 22:10:05.950388 14953 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:05.950472 14953 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
W20260923 22:10:05.950767 14953 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:10:06.048919 14957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:06.049220 14957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:06.049263 14957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:06.050984 14957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:06.051049 14957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:06.051092 14957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:10:06.053284 14957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:41023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:39531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=40017
--tserver_master_addrs=127.1.244.254:41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:06.053541 14957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:06.053857 14957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:06.059449 14963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:06.059568 14957 server_base.cc:1034] running on GCE node
W20260923 22:10:06.059729 14962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:06.061370 14965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:06.061661 14957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:06.061942 14957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:06.063122 14957 hybrid_clock.cc:648] HybridClock initialized: now 1790201406063104 us; error 30 us; skew 500 ppm
Sep 23 22:10:06 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:06.066258 14957 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:10:06.066803 14957 webserver.cc:460] Webserver started at http://127.1.244.194:40017/ using document root <none> and password file <none>
I20260923 22:10:06.067065 14957 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:06.067147 14957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:10:06 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:10:06.093163 14957 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:06.094275 14973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:06.094449 14957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:06.094547 14957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "ed524fe4d67544129dfd95bd6bb03af9"
format_stamp: "Formatted at 2026-09-23 22:10:05 on dist-test-slave-b5jz"
server_key: "d90fd62910a25063d5ea361788e85e3f"
server_key_iv: "07055a6dc23e1943eb35776314de3ec8"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:06.095036 14957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:06.111454 14957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:06.113713 14957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:06.113878 14957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:06.114143 14957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:06.114543 14957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:06.114598 14957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:06.114629 14957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:06.114648 14957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:06.122222 14957 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:39531
I20260923 22:10:06.122345 15086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:39531 every 8 connection(s)
I20260923 22:10:06.122725 14957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:06.126894 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 14957
I20260923 22:10:06.127185 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:41023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:42157
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:41929
--webserver_port=37923
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:41023/kms with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:10:06 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:06.137454 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:06.123522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:42523 (local address 127.1.244.254:41929)
0923 22:10:06.123616 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:06.123618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:06.123810 (+ 192us) server_negotiation.cc:407] Connection header received
0923 22:10:06.124368 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:06.124369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:06.124384 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:06.124465 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:06.125177 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.125940 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.127457 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.127781 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.127832 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:06.133512 (+ 5680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:06.133514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:06.133514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:06.133523 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:06.135566 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:06.136159 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:06.136160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:06.136160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:06.136226 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:06.136555 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:06.136556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:06.136557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:06.136713 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:06.136764 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:06.137212 (+ 448us) server_negotiation.cc:299] Negotiation successful
0923 22:10:06.137315 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 22:10:06.138504 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:06.123484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41929 (local address 127.1.244.194:42523)
0923 22:10:06.123666 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:06.123711 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:06.124243 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:06.124556 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:06.124560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:06.124788 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:06.125059 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:06.125065 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.126055 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.126056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:06.127352 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:06.127356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.127945 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.127946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:06.128085 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:06.131723 (+ 3638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:06.131735 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:06.133387 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:06.135676 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:06.135684 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:06.135687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:06.136041 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:06.136310 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:06.136311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:06.136311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:06.136467 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:06.136817 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:06.136823 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:06.137017 (+ 194us) client_negotiation.cc:769] Sending connection context
0923 22:10:06.138004 (+ 987us) client_negotiation.cc:240] Negotiation successful
0923 22:10:06.138214 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
I20260923 22:10:06.139299 15087 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41929
I20260923 22:10:06.139443 15087 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:06.139691 15087 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
W20260923 22:10:06.140302 15087 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:41929 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:10:06.140376 15087 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:06.140432 15087 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
I20260923 22:10:06.140769 15087 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:06.140837 15087 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
W20260923 22:10:06.141146 15087 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:10:06.230317 15090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:06.230530 15090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:06.230564 15090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:06.231769 15090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:06.231829 15090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:06.231874 15090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:10:06.233438 15090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:41023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:42157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=37923
--tserver_master_addrs=127.1.244.254:41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:06.233691 15090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:06.233902 15090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:06.239070 15096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:06.239184 15098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:06.239388 15095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:06.239774 15090 server_base.cc:1034] running on GCE node
I20260923 22:10:06.239969 15090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:06.240242 15090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:06.243510 15090 hybrid_clock.cc:648] HybridClock initialized: now 1790201406242391 us; error 1136 us; skew 500 ppm
Sep 23 22:10:06 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:06.246328 15090 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:10:06.246835 15090 webserver.cc:460] Webserver started at http://127.1.244.195:37923/ using document root <none> and password file <none>
I20260923 22:10:06.247116 15090 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:06.247203 15090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:10:06 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:10:06.278837 15090 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260923 22:10:06.280102 15107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:06.280332 15090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:06.280417 15090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "252330e68d594f1d9704b637466cb393"
format_stamp: "Formatted at 2026-09-23 22:10:05 on dist-test-slave-b5jz"
server_key: "2824cc896ef799626fd56559f317db5e"
server_key_iv: "85b53e83153259b30971a08bcb9285a5"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:06.280882 15090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:06.307176 15090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:06.309156 15090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:06.309346 15090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:06.309612 15090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:06.310010 15090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:06.310056 15090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:06.310094 15090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:06.310119 15090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:06.317315 15090 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:42157
I20260923 22:10:06.317699 15090 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:06.318310 15220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:42157 every 8 connection(s)
I20260923 22:10:06.323648 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 15090
Sep 23 22:10:06 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:06.329146 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:06.318770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:40285 (local address 127.1.244.254:41929)
0923 22:10:06.318841 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:06.318843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:06.318853 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:06.319384 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:06.319385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:06.319399 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:06.319432 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:06.320270 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.321007 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.322408 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.322731 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.322780 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:06.325601 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:06.325602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:06.325603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:06.325611 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:06.327399 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:06.327942 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:06.327943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:06.327944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:06.328012 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:06.328361 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:06.328362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:06.328363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:06.328504 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:06.328561 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:06.328956 (+ 395us) server_negotiation.cc:299] Negotiation successful
0923 22:10:06.329045 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 22:10:06.329305 15223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:06.318439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41929 (local address 127.1.244.195:40285)
0923 22:10:06.318605 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:06.318648 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:06.319237 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:06.319647 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:06.319652 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:06.319874 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:06.320138 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:06.320144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.321117 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.321118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:06.322294 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:06.322297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.322891 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.322892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:06.323039 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:06.323929 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:06.323939 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:06.325477 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:06.327495 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:06.327502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:06.327505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:06.327858 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:06.328102 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:06.328103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:06.328103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:06.328264 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:06.328598 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:06.328605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:06.328774 (+ 169us) client_negotiation.cc:769] Sending connection context
0923 22:10:06.328895 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 22:10:06.329068 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":99,"threads_started":1}
I20260923 22:10:06.329909 15221 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41929
I20260923 22:10:06.330052 15221 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:06.330300 15221 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
W20260923 22:10:06.330899 15221 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:41929 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:10:06.330969 15221 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:06.331023 15221 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
I20260923 22:10:06.331321 15221 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:06.331405 15221 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
W20260923 22:10:06.331728 15221 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:10:06.335860 15225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:06.324210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.1.244.254:41929)
0923 22:10:06.327623 (+ 3413us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:06.327625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:06.327633 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:06.327654 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:06.327655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:06.327668 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:06.327747 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:06.329426 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.330193 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.331291 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:06.331733 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:06.331849 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:06.333037 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:06.333039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:06.333039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:06.333046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:06.334701 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:06.335021 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:06.335023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:06.335023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:06.335088 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:06.335287 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:06.335288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:06.335288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:06.335420 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:06.335627 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:06.335669 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 22:10:06.335753 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3391,"thread_start_us":78,"threads_started":1}
Sep 23 22:10:06 dist-test-slave-b5jz krb5kdc[13759](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:06.950948 14953 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:06.951128 14953 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
I20260923 22:10:06.951608 14755 ts_manager.cc:194] Registered new tserver with Master: b4f932933fba41fea005ed4357383091 (127.1.244.193:46415)
I20260923 22:10:06.952328 14755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:53305
I20260923 22:10:07.141347 15087 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:07.141511 15087 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
I20260923 22:10:07.141909 14755 ts_manager.cc:194] Registered new tserver with Master: ed524fe4d67544129dfd95bd6bb03af9 (127.1.244.194:39531)
I20260923 22:10:07.142328 14755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:42523
Sep 23 22:10:07 dist-test-slave-b5jz krb5kdc[13759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260923 22:10:07.331929 15221 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:07.332083 15221 heartbeater.cc:511] Master 127.1.244.254:41929 requested a full tablet report, sending...
I20260923 22:10:07.332430 14754 ts_manager.cc:194] Registered new tserver with Master: 252330e68d594f1d9704b637466cb393 (127.1.244.195:42157)
I20260923 22:10:07.332942 14754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:40285
I20260923 22:10:07.343143 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:07.352682 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.344355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.1.244.254:41929)
0923 22:10:07.344706 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:07.344708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:07.344811 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:10:07.344909 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:07.344910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:07.344923 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:07.344966 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:07.345743 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.346549 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.347689 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.348002 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.348046 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.349612 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:07.349614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:07.349614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:07.349621 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:07.351382 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:07.351819 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:07.351821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:07.351821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:07.351882 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:07.352068 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:07.352069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:07.352069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:07.352198 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:07.352268 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:07.352463 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 22:10:07.352544 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":79,"threads_started":1}
I20260923 22:10:07.355331 14754 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:07.358121 14754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:07.375856 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.365973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:39531 (local address 127.0.0.1:46712)
0923 22:10:07.371245 (+ 5272us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:07.371252 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:07.371282 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:07.372023 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:07.372024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:07.372026 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:07.372071 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.372073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.373363 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.373364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.374535 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.374537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.375587 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.375588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.375656 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.375660 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:07.375713 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:10:07.375756 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5259,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260923 22:10:07.375913 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.366060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.1.244.193:46415)
0923 22:10:07.371382 (+ 5322us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:07.371387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:07.371396 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:10:07.371431 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:07.371433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:07.371448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:07.371511 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:07.371796 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.372812 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.374617 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.375334 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.375400 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.375456 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:07.375801 (+ 345us) server_negotiation.cc:299] Negotiation successful
0923 22:10:07.375843 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5286,"thread_start_us":55,"threads_started":1}
I20260923 22:10:07.376238 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.366417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.1.244.194:39531)
0923 22:10:07.371747 (+ 5330us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:07.371752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:07.371767 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:10:07.371806 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:07.371808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:07.371825 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:07.371916 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:07.372171 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.373265 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.374632 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.375408 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.375480 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.375534 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:07.376127 (+ 593us) server_negotiation.cc:299] Negotiation successful
0923 22:10:07.376169 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5282,"thread_start_us":52,"threads_started":1}
I20260923 22:10:07.376039 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.365956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:46415 (local address 127.0.0.1:48252)
0923 22:10:07.371033 (+ 5077us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:07.371092 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:07.371153 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:07.371645 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:07.371648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:07.371650 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:07.371722 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.371724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.372905 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.372906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.374524 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.374527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.375490 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.375491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.375599 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.375614 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:10:07.375678 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:10:07.375729 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5039,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260923 22:10:07.380198 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.366207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42157 (local address 127.0.0.1:42678)
0923 22:10:07.373505 (+ 7298us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:07.373515 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:07.373565 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:07.376561 (+ 2996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:07.376563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:07.376565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:07.376604 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.376605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.378185 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.378186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.379065 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.379068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.379941 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.379942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.380006 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.380009 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:07.380063 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:10:07.380107 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7280,"thread_start_us":80,"threads_started":1}
I20260923 22:10:07.380604 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.366323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.1.244.195:42157)
0923 22:10:07.376388 (+ 10065us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:07.376392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:07.376403 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:10:07.376438 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:07.376440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:07.376456 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:07.376675 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:07.376765 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.378081 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.379160 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.379839 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.380391 (+ 552us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.380457 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:07.380483 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:10:07.380526 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10026,"thread_start_us":42,"threads_started":1}
I20260923 22:10:07.380537 15021 tablet_service.cc:1467] Processing CreateTablet for tablet a14313bcd5f544d191e7c0448fe674fc (DEFAULT_TABLE table=test-table [id=344dea1c818c4da2b5d05ea42992d182]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:07.380961 15021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a14313bcd5f544d191e7c0448fe674fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:07.381299 15155 tablet_service.cc:1467] Processing CreateTablet for tablet a14313bcd5f544d191e7c0448fe674fc (DEFAULT_TABLE table=test-table [id=344dea1c818c4da2b5d05ea42992d182]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:07.381589 15155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a14313bcd5f544d191e7c0448fe674fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:07.384016 14887 tablet_service.cc:1467] Processing CreateTablet for tablet a14313bcd5f544d191e7c0448fe674fc (DEFAULT_TABLE table=test-table [id=344dea1c818c4da2b5d05ea42992d182]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:07.384492 14887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a14313bcd5f544d191e7c0448fe674fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:07.388206 15255 tablet_bootstrap.cc:492] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: Bootstrap starting.
I20260923 22:10:07.388300 15256 tablet_bootstrap.cc:492] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091: Bootstrap starting.
I20260923 22:10:07.388963 15256 tablet_bootstrap.cc:654] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:07.389281 15255 tablet_bootstrap.cc:654] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:07.389333 15256 log.cc:824] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:07.389631 15255 log.cc:824] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:07.389748 15254 tablet_bootstrap.cc:492] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: Bootstrap starting.
I20260923 22:10:07.390393 15254 tablet_bootstrap.cc:654] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:07.390455 15256 tablet_bootstrap.cc:492] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091: No bootstrap required, opened a new log
I20260923 22:10:07.390556 15256 ts_tablet_manager.cc:1397] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:07.390753 15254 log.cc:824] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:07.391662 15255 tablet_bootstrap.cc:492] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: No bootstrap required, opened a new log
I20260923 22:10:07.391733 15255 ts_tablet_manager.cc:1397] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260923 22:10:07.393966 15256 raft_consensus.cc:348] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } }
I20260923 22:10:07.394688 15256 raft_consensus.cc:374] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:07.394788 15256 raft_consensus.cc:729] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f932933fba41fea005ed4357383091, State: Initialized, Role: FOLLOWER
I20260923 22:10:07.395001 15256 consensus_queue.cc:260] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } }
I20260923 22:10:07.395402 15256 ts_tablet_manager.cc:1428] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 22:10:07.394933 15255 raft_consensus.cc:348] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } }
I20260923 22:10:07.395582 14953 heartbeater.cc:503] Master 127.1.244.254:41929 was elected leader, sending a full tablet report...
I20260923 22:10:07.395576 15255 raft_consensus.cc:374] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:07.395704 15255 raft_consensus.cc:729] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed524fe4d67544129dfd95bd6bb03af9, State: Initialized, Role: FOLLOWER
I20260923 22:10:07.395888 15255 consensus_queue.cc:260] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } }
I20260923 22:10:07.396217 15255 ts_tablet_manager.cc:1428] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260923 22:10:07.396387 15087 heartbeater.cc:503] Master 127.1.244.254:41929 was elected leader, sending a full tablet report...
I20260923 22:10:07.396567 15254 tablet_bootstrap.cc:492] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: No bootstrap required, opened a new log
I20260923 22:10:07.396651 15254 ts_tablet_manager.cc:1397] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260923 22:10:07.398367 15254 raft_consensus.cc:348] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } }
I20260923 22:10:07.398514 15254 raft_consensus.cc:374] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:07.398556 15254 raft_consensus.cc:729] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 252330e68d594f1d9704b637466cb393, State: Initialized, Role: FOLLOWER
I20260923 22:10:07.398672 15254 consensus_queue.cc:260] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } }
I20260923 22:10:07.398931 15254 ts_tablet_manager.cc:1428] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:10:07.399072 15221 heartbeater.cc:503] Master 127.1.244.254:41929 was elected leader, sending a full tablet report...
I20260923 22:10:07.427651 15262 raft_consensus.cc:482] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:07.427949 15262 raft_consensus.cc:504] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } }
I20260923 22:10:07.429145 15262 leader_election.cc:288] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f932933fba41fea005ed4357383091 (127.1.244.193:46415), ed524fe4d67544129dfd95bd6bb03af9 (127.1.244.194:39531)
W20260923 22:10:07.430975 14954 tablet.cc:2369] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:07.433164 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.428726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:32947 (local address 127.1.244.193:46415)
0923 22:10:07.428774 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:07.428777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:07.429378 (+ 601us) server_negotiation.cc:407] Connection header received
0923 22:10:07.429481 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:07.429482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:07.429493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:07.429522 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:07.429746 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.430480 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.432079 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.432654 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.432703 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.432712 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:07.432984 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 22:10:07.433028 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:10:07.433604 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.428649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:46415 (local address 127.1.244.195:32947)
0923 22:10:07.429309 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:07.429320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:07.429408 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:07.429599 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:07.429600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:07.429602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:07.429676 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.429677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.430598 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.430599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.431971 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.431974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.432789 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.432790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.432857 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.432861 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:07.433451 (+ 590us) client_negotiation.cc:240] Negotiation successful
0923 22:10:07.433503 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":108,"threads_started":1}
I20260923 22:10:07.434111 14903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a14313bcd5f544d191e7c0448fe674fc" candidate_uuid: "252330e68d594f1d9704b637466cb393" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f932933fba41fea005ed4357383091" is_pre_election: true
I20260923 22:10:07.434297 14903 raft_consensus.cc:2454] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 252330e68d594f1d9704b637466cb393 in term 0.
I20260923 22:10:07.435400 15110 leader_election.cc:302] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 252330e68d594f1d9704b637466cb393, b4f932933fba41fea005ed4357383091; no voters:
I20260923 22:10:07.435611 15262 raft_consensus.cc:2781] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:07.435663 15262 raft_consensus.cc:482] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:07.435685 15262 raft_consensus.cc:3037] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:07.436600 15262 raft_consensus.cc:504] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } }
I20260923 22:10:07.436905 14903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a14313bcd5f544d191e7c0448fe674fc" candidate_uuid: "252330e68d594f1d9704b637466cb393" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f932933fba41fea005ed4357383091"
I20260923 22:10:07.437005 15262 leader_election.cc:288] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [CANDIDATE]: Term 1 election: Requested vote from peers b4f932933fba41fea005ed4357383091 (127.1.244.193:46415), ed524fe4d67544129dfd95bd6bb03af9 (127.1.244.194:39531)
I20260923 22:10:07.436986 14903 raft_consensus.cc:3037] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:07.437870 14903 raft_consensus.cc:2454] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 252330e68d594f1d9704b637466cb393 in term 1.
I20260923 22:10:07.438057 15110 leader_election.cc:302] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 252330e68d594f1d9704b637466cb393, b4f932933fba41fea005ed4357383091; no voters:
I20260923 22:10:07.438179 15262 raft_consensus.cc:2781] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:07.438318 15262 raft_consensus.cc:686] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 1 LEADER]: Becoming Leader. State: Replica: 252330e68d594f1d9704b637466cb393, State: Running, Role: LEADER
I20260923 22:10:07.438402 15262 consensus_queue.cc:237] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } }
I20260923 22:10:07.439363 14754 catalog_manager.cc:5495] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 reported cstate change: term changed from 0 to 1, leader changed from <none> to 252330e68d594f1d9704b637466cb393 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "252330e68d594f1d9704b637466cb393" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "252330e68d594f1d9704b637466cb393" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42157 } health_report { overall_health: HEALTHY } } }
I20260923 22:10:07.443089 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.429085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:54785 (local address 127.1.244.194:39531)
0923 22:10:07.429126 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:07.429128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:07.434063 (+ 4935us) server_negotiation.cc:407] Connection header received
0923 22:10:07.434171 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:07.434172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:07.434183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:07.434215 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:07.434390 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.435120 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.440143 (+ 5023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.440747 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.440801 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.440810 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:07.442947 (+ 2137us) server_negotiation.cc:299] Negotiation successful
0923 22:10:07.443005 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:10:07.443496 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.428913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:39531 (local address 127.1.244.195:54785)
0923 22:10:07.433995 (+ 5082us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:07.434007 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:07.434093 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:07.434279 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:07.434280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:07.434281 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:07.434314 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.434315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.437113 (+ 2798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.437114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.440026 (+ 2912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:07.440029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.442789 (+ 2760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.442790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:07.442858 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.442861 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:07.443333 (+ 472us) client_negotiation.cc:240] Negotiation successful
0923 22:10:07.443380 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5069,"thread_start_us":64,"threads_started":1}
I20260923 22:10:07.444193 15041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a14313bcd5f544d191e7c0448fe674fc" candidate_uuid: "252330e68d594f1d9704b637466cb393" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed524fe4d67544129dfd95bd6bb03af9" is_pre_election: true
I20260923 22:10:07.444350 15041 raft_consensus.cc:2454] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 252330e68d594f1d9704b637466cb393 in term 0.
I20260923 22:10:07.444460 15040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a14313bcd5f544d191e7c0448fe674fc" candidate_uuid: "252330e68d594f1d9704b637466cb393" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed524fe4d67544129dfd95bd6bb03af9"
I20260923 22:10:07.444507 15040 raft_consensus.cc:3037] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:07.445312 15040 raft_consensus.cc:2454] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 252330e68d594f1d9704b637466cb393 in term 1.
I20260923 22:10:07.448660 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.444688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42684 (local address 127.1.244.195:42157)
0923 22:10:07.445272 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:07.445274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:07.445283 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:10:07.445309 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:07.445310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:07.445326 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:07.445397 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:07.445586 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.446352 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.447609 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.447882 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.447942 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.448159 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:07.448276 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:07.448346 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:07.448573 (+ 227us) server_negotiation.cc:299] Negotiation successful
0923 22:10:07.448594 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563}
I20260923 22:10:07.450865 15040 raft_consensus.cc:1261] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [term 1 FOLLOWER]: Refusing update from remote peer 252330e68d594f1d9704b637466cb393: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:07.451202 14903 raft_consensus.cc:1261] T a14313bcd5f544d191e7c0448fe674fc P b4f932933fba41fea005ed4357383091 [term 1 FOLLOWER]: Refusing update from remote peer 252330e68d594f1d9704b637466cb393: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:07.451709 15262 consensus_queue.cc:1035] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed524fe4d67544129dfd95bd6bb03af9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 39531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:07.451854 15262 consensus_queue.cc:1035] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f932933fba41fea005ed4357383091" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 46415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:07.458551 15268 mvcc.cc:205] Tried to move back new op lower bound from 7332664964915437568 to 7332664964868149248. Current Snapshot: MvccSnapshot[applied={T|T < 7332664964915437568}]
I20260923 22:10:07.458810 15271 mvcc.cc:205] Tried to move back new op lower bound from 7332664964915437568 to 7332664964868149248. Current Snapshot: MvccSnapshot[applied={T|T < 7332664964915437568}]
I20260923 22:10:07.460619 15269 mvcc.cc:205] Tried to move back new op lower bound from 7332664964915437568 to 7332664964868149248. Current Snapshot: MvccSnapshot[applied={T|T < 7332664964915437568}]
I20260923 22:10:07.461964 14754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:07.462057 14754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:07.462890 14754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec885e2a18ae4095926d44bf529ba131: Sending DeleteTablet for 3 replicas of tablet a14313bcd5f544d191e7c0448fe674fc
I20260923 22:10:07.463434 14887 tablet_service.cc:1514] Processing DeleteTablet for tablet a14313bcd5f544d191e7c0448fe674fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:07 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:48252
I20260923 22:10:07.463677 15021 tablet_service.cc:1514] Processing DeleteTablet for tablet a14313bcd5f544d191e7c0448fe674fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:07 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46712
I20260923 22:10:07.463882 15155 tablet_service.cc:1514] Processing DeleteTablet for tablet a14313bcd5f544d191e7c0448fe674fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:07 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:42678
I20260923 22:10:07.464646 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 14793
I20260923 22:10:07.468247 15279 tablet_replica.cc:330] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: stopping tablet replica
I20260923 22:10:07.468400 15280 tablet_replica.cc:330] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: stopping tablet replica
I20260923 22:10:07.472424 15280 raft_consensus.cc:2229] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:10:07.472628 15279 raft_consensus.cc:2229] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:07.472841 15280 raft_consensus.cc:2258] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:07.473003 15279 raft_consensus.cc:2258] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:07.473615 15279 ts_tablet_manager.cc:1905] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:07.474057 15280 ts_tablet_manager.cc:1905] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:07.477427 15279 ts_tablet_manager.cc:1918] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:07.477581 15279 log.cc:1196] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/a14313bcd5f544d191e7c0448fe674fc
I20260923 22:10:07.477767 15280 ts_tablet_manager.cc:1918] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:07.478185 15279 ts_tablet_manager.cc:1939] T a14313bcd5f544d191e7c0448fe674fc P ed524fe4d67544129dfd95bd6bb03af9: Deleting consensus metadata
I20260923 22:10:07.478307 15280 log.cc:1196] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/a14313bcd5f544d191e7c0448fe674fc
I20260923 22:10:07.478690 15280 ts_tablet_manager.cc:1939] T a14313bcd5f544d191e7c0448fe674fc P 252330e68d594f1d9704b637466cb393: Deleting consensus metadata
I20260923 22:10:07.479249 14739 catalog_manager.cc:4841] TS ed524fe4d67544129dfd95bd6bb03af9 (127.1.244.194:39531): tablet a14313bcd5f544d191e7c0448fe674fc (table test-table [id=344dea1c818c4da2b5d05ea42992d182]) successfully deleted
I20260923 22:10:07.479837 14740 catalog_manager.cc:4841] TS 252330e68d594f1d9704b637466cb393 (127.1.244.195:42157): tablet a14313bcd5f544d191e7c0448fe674fc (table test-table [id=344dea1c818c4da2b5d05ea42992d182]) successfully deleted
W20260923 22:10:07.480232 14741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:46415 (error 108)
I20260923 22:10:07.483984 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 14957
I20260923 22:10:07.490777 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.490632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:46415 (local address 127.0.0.1:48256)
0923 22:10:07.490690 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:07.490735 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:46415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 22:10:07.491044 14741 catalog_manager.cc:4584] TS b4f932933fba41fea005ed4357383091 (127.1.244.193:46415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a14313bcd5f544d191e7c0448fe674fc: Network error: Client connection negotiation failed: client connection to 127.1.244.193:46415: connect: Connection refused (error 111)
I20260923 22:10:07.491322 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 15090
I20260923 22:10:07.499388 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 14722
I20260923 22:10:07.506461 2003 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 22:10:07.851019 2003 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 22:10:07.851111 2003 mini_ranger.cc:66] Stopping Ranger...
I20260923 22:10:08.195811 2003 mini_ranger.cc:68] Stopped Ranger
2026-09-23 22:10:08.195 UTC [13789] LOG: received smart shutdown request
2026-09-23 22:10:08.199 UTC [13789] LOG: background worker "logical replication launcher" (PID 13798) exited with exit code 1
2026-09-23 22:10:08.200 UTC [13793] LOG: shutting down
2026-09-23 22:10:08.361 UTC [13789] LOG: database system is shut down
2026-09-23T22:10:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47047 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:10:08 dist-test-slave-b5jz krb5kdc[15290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:10:08 dist-test-slave-b5jz krb5kdc[15290](info): set up 2 sockets
Sep 23 22:10:08 dist-test-slave-b5jz krb5kdc[15290](info): commencing operation
krb5kdc: starting...
W20260923 22:10:09.511490 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:10:09 dist-test-slave-b5jz krb5kdc[15290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:10:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:10:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:10:09.630256 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38437
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38437,127.1.244.253:38479,127.1.244.252:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:09.711748 15303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:09.711971 15303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:09.711995 15303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:09.713272 15303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:09.713320 15303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:09.713336 15303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:09.713382 15303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:09.713411 15303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:09.715174 15303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38437,127.1.244.253:38479,127.1.244.252:43167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:09.715454 15303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:09.715782 15303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:09.720999 15308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:09.721145 15309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:09.721145 15311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:09.721171 15303 server_base.cc:1034] running on GCE node
I20260923 22:10:09.721539 15303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:09.721840 15303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:09.722985 15303 hybrid_clock.cc:648] HybridClock initialized: now 1790201409722968 us; error 33 us; skew 500 ppm
Sep 23 22:10:09 dist-test-slave-b5jz krb5kdc[15290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:09.725622 15303 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:10:09.725992 15303 webserver.cc:460] Webserver started at http://127.1.244.254:34009/ using document root <none> and password file <none>
I20260923 22:10:09.726193 15303 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:09.726234 15303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:09.726322 15303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:09.727317 15303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "3188c5bd3fd64db5bd46ca4f235a7c06"
format_stamp: "Formatted at 2026-09-23 22:10:09 on dist-test-slave-b5jz"
server_key: "f536576b479c032d5d8989c7a4e9bdeb"
server_key_iv: "ce2622a84fa71a8098ceaeec321b3964"
server_key_version: "encryptionkey@0"
I20260923 22:10:09.727700 15303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "3188c5bd3fd64db5bd46ca4f235a7c06"
format_stamp: "Formatted at 2026-09-23 22:10:09 on dist-test-slave-b5jz"
server_key: "f536576b479c032d5d8989c7a4e9bdeb"
server_key_iv: "ce2622a84fa71a8098ceaeec321b3964"
server_key_version: "encryptionkey@0"
I20260923 22:10:09.728961 15303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:09.729671 15318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:09.729852 15303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:10:09.729938 15303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "3188c5bd3fd64db5bd46ca4f235a7c06"
format_stamp: "Formatted at 2026-09-23 22:10:09 on dist-test-slave-b5jz"
server_key: "f536576b479c032d5d8989c7a4e9bdeb"
server_key_iv: "ce2622a84fa71a8098ceaeec321b3964"
server_key_version: "encryptionkey@0"
I20260923 22:10:09.729992 15303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:09.745702 15303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:09.748091 15303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:09.748207 15303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:09.751965 15303 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:38437
I20260923 22:10:09.752055 15370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:38437 every 8 connection(s)
I20260923 22:10:09.752403 15303 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:09.753324 15371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:09.754907 15371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:09.755100 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 15303
I20260923 22:10:09.755210 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:10:09.755409 2003 external_mini_cluster.cc:1428] Setting key df1c7d416db6290777a3a3ed8ec397c1
Sep 23 22:10:09 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.253@KRBTEST.COM" created.
I20260923 22:10:09.766079 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.755698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38437 (local address 127.0.0.1:36888)
0923 22:10:09.755872 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.755912 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:09.756382 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:09.756715 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:09.756718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:09.757093 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:09.757336 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.757342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.758739 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.758740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.759496 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.759499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.760329 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.760330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.760400 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.761047 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:09.761055 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:09.762565 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:09.764480 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.764486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.764488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.764740 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.764959 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.764960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.764960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.765097 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.765441 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:09.765448 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:09.765597 (+ 149us) client_negotiation.cc:769] Sending connection context
0923 22:10:09.765697 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 22:10:09.765885 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260923 22:10:09.766093 15374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.755913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.1.244.254:38437)
0923 22:10:09.756072 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:09.756077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:09.756117 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:10:09.756538 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:09.756540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:09.756571 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:09.756697 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:09.757439 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.758642 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.759643 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.760206 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.760337 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.762659 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:09.762667 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:09.762668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:09.762675 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:09.764394 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.764816 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.764818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.764818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.764897 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.765159 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.765160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.765160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.765358 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:09.765422 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:09.765763 (+ 341us) server_negotiation.cc:299] Negotiation successful
0923 22:10:09.765974 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260923 22:10:09.766922 15371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:09.767307 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.767183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38479 (local address 127.0.0.1:49514)
0923 22:10:09.767239 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.767287 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260923 22:10:09.767472 15319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
I20260923 22:10:09.767942 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.767853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38479 (local address 127.0.0.1:49526)
0923 22:10:09.767900 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.767925 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260923 22:10:09.768071 15371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:38479: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
I20260923 22:10:09.781888 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:38479
--webserver_interface=127.1.244.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38437,127.1.244.253:38479,127.1.244.252:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:09.817329 15371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } attempt: 1
I20260923 22:10:09.817932 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.817745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38479 (local address 127.0.0.1:49530)
0923 22:10:09.817826 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.817883 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260923 22:10:09.818365 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.818233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38479 (local address 127.0.0.1:49534)
0923 22:10:09.818291 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.818329 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:10:09.818519 15371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:38479: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
W20260923 22:10:09.863646 15378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:09.863862 15378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:09.863884 15378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:09.865043 15378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:09.865087 15378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:09.865100 15378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:09.865113 15378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:09.865166 15378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:09.866657 15378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38437,127.1.244.253:38479,127.1.244.252:43167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:38479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.244.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:09.866910 15378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:09.867136 15378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:09.871857 15383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:09.871906 15384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:09.871966 15386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:09.872205 15378 server_base.cc:1034] running on GCE node
I20260923 22:10:09.872462 15378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:09.872751 15378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:09.873898 15378 hybrid_clock.cc:648] HybridClock initialized: now 1790201409873889 us; error 31 us; skew 500 ppm
Sep 23 22:10:09 dist-test-slave-b5jz krb5kdc[15290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:09.876410 15378 init.cc:377] Logged in from keytab as kudu/127.1.244.253@KRBTEST.COM (short username kudu)
I20260923 22:10:09.876842 15378 webserver.cc:460] Webserver started at http://127.1.244.253:34157/ using document root <none> and password file <none>
I20260923 22:10:09.877064 15378 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:09.877132 15378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:09.877282 15378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:09.878310 15378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/data/instance:
uuid: "975263546f574b7a9c24599024caba81"
format_stamp: "Formatted at 2026-09-23 22:10:09 on dist-test-slave-b5jz"
server_key: "1bb8e119ef5162f347bfd3e89aef4d56"
server_key_iv: "e0d708f2aea0f406ad59c5d0953a706c"
server_key_version: "encryptionkey@0"
I20260923 22:10:09.878672 15378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/wal/instance:
uuid: "975263546f574b7a9c24599024caba81"
format_stamp: "Formatted at 2026-09-23 22:10:09 on dist-test-slave-b5jz"
server_key: "1bb8e119ef5162f347bfd3e89aef4d56"
server_key_iv: "e0d708f2aea0f406ad59c5d0953a706c"
server_key_version: "encryptionkey@0"
I20260923 22:10:09.880045 15378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:10:09.880718 15393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:09.880883 15378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:10:09.880980 15378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/wal
uuid: "975263546f574b7a9c24599024caba81"
format_stamp: "Formatted at 2026-09-23 22:10:09 on dist-test-slave-b5jz"
server_key: "1bb8e119ef5162f347bfd3e89aef4d56"
server_key_iv: "e0d708f2aea0f406ad59c5d0953a706c"
server_key_version: "encryptionkey@0"
I20260923 22:10:09.881062 15378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:09.886657 15371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } attempt: 2
I20260923 22:10:09.887046 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.886921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38479 (local address 127.0.0.1:49540)
0923 22:10:09.886980 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.887016 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:10:09.887372 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.887273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38479 (local address 127.0.0.1:49546)
0923 22:10:09.887319 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.887342 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:10:09.887456 15371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:38479: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38479: connect: Connection refused (error 111)
I20260923 22:10:09.900791 15378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:09.903204 15378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:09.903389 15378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:09.907418 15378 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:38479
I20260923 22:10:09.907466 15445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:38479 every 8 connection(s)
I20260923 22:10:09.907871 15378 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
I20260923 22:10:09.908701 15446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:09.910320 15446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:10:09 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:09.916465 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 15378
I20260923 22:10:09.916579 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-1/wal/instance
I20260923 22:10:09.916764 2003 external_mini_cluster.cc:1428] Setting key 3192cb33c57b48d96d95f9c2b0c5677c
I20260923 22:10:09.919222 15374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.911052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36904 (local address 127.1.244.254:38437)
0923 22:10:09.911121 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:09.911123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:09.911290 (+ 167us) server_negotiation.cc:407] Connection header received
0923 22:10:09.911793 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:09.911794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:09.911806 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:09.911872 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:09.912437 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.912902 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.913831 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.914017 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.914071 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.916172 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:09.916174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:09.916174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:09.916181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:09.917793 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.918296 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.918297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.918298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.918371 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.918606 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.918607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.918608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.918756 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:09.918827 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:09.919060 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 22:10:09.919151 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:10:09.919333 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.911018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38437 (local address 127.0.0.1:36904)
0923 22:10:09.911158 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.911200 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:09.911703 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:09.911927 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:09.911930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:09.912174 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:09.912365 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.912369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.912978 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.912978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.913754 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.913756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.914090 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.914091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.914190 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.914714 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:09.914722 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:09.916085 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:09.917877 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.917884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.917887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.918211 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.918445 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.918446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.918446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.918543 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.918822 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:09.918826 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:09.918933 (+ 107us) client_negotiation.cc:769] Sending connection context
0923 22:10:09.919014 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:10:09.919129 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260923 22:10:09.919890 15446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:10:09 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.252@KRBTEST.COM; defaulting to no policy
I20260923 22:10:09.927209 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.920151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38479 (local address 127.0.0.1:49556)
0923 22:10:09.920205 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.920217 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:09.920260 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:09.920501 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:09.920502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:09.920669 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:09.920733 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.920735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.921559 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.921560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.922464 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.922466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.922899 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.922901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.922977 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.923331 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:09.923336 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:09.924598 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:09.926281 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.926282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.926283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.926466 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.926646 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.926647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.926647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.926700 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.926932 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:09.926933 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:09.926964 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 22:10:09.927046 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 22:10:09.927137 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 22:10:09.927320 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.920207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.1.244.253:38479)
0923 22:10:09.920353 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:09.920358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:09.920376 (+ 18us) server_negotiation.cc:407] Connection header received
0923 22:10:09.920404 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:09.920406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:09.920417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:09.920480 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:09.920806 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.921467 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.922548 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.922825 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.922877 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.924693 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:09.924702 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:09.924702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:09.924710 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:09.926140 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.926535 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.926538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.926538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.926587 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.926753 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.926754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.926754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.926877 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:09.926936 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:09.927070 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 22:10:09.927208 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.252@KRBTEST.COM" created.
I20260923 22:10:09.927932 15446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:09.928277 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.928159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60584)
0923 22:10:09.928212 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.928255 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 22:10:09.928438 15396 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
I20260923 22:10:09.928819 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.928734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60592)
0923 22:10:09.928782 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.928805 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:10:09.928938 15446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43167: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
I20260923 22:10:09.942123 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:43167
--webserver_interface=127.1.244.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38437,127.1.244.253:38479,127.1.244.252:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:09.978169 15446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } attempt: 1
I20260923 22:10:09.978590 15371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } attempt: 3
I20260923 22:10:09.978753 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.978586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60596)
0923 22:10:09.978662 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.978714 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260923 22:10:09.979219 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.979078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60602)
0923 22:10:09.979141 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.979167 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 22:10:09.979333 15446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43167: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Sep 23 22:10:09 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260923 22:10:09.986145 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.978944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.1.244.253:38479)
0923 22:10:09.978981 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:09.978983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:09.979110 (+ 127us) server_negotiation.cc:407] Connection header received
0923 22:10:09.979136 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:09.979138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:09.979149 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:09.979206 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:09.979654 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.980435 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.981129 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.981447 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.981519 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.983446 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:09.983447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:09.983448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:09.983454 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:09.984962 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.985372 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.985373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.985373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.985424 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.985682 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.985684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.985684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.985802 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:09.985867 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:09.986019 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 22:10:09.986085 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:10:09.986197 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.978901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38479 (local address 127.0.0.1:49560)
0923 22:10:09.978958 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.978970 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:09.979033 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:09.979235 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:09.979236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:09.979437 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:09.979517 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.979520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.980534 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.980535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.981050 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.981052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.981522 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.981523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.981589 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.982077 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:09.982081 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:09.983362 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:09.985038 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.985039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.985039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.985282 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.985504 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.985504 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.985505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.985574 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.985877 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:09.985878 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:09.985928 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 22:10:09.985999 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:10:09.986097 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:10:09.986620 15371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:09.987012 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.986901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60618)
0923 22:10:09.986956 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.986989 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:10:09.987351 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.987253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60622)
0923 22:10:09.987304 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.987330 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:10:09.987466 15371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43167: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
I20260923 22:10:10.018603 15371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } attempt: 1
I20260923 22:10:10.019083 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.018947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60632)
0923 22:10:10.019021 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.019053 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 22:10:10.019467 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.019370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60648)
0923 22:10:10.019416 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.019448 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:10:10.019569 15371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43167: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
W20260923 22:10:10.025278 15453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:10.025458 15453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:10.025480 15453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:10.026597 15453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:10.026636 15453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:10.026652 15453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:10.026692 15453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:10.026721 15453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:10.028249 15453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38437,127.1.244.253:38479,127.1.244.252:43167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:43167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.244.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:10.028501 15453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:10.028721 15453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:10.033375 15458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:10.033336 15459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:10.033360 15461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:10.033630 15453 server_base.cc:1034] running on GCE node
I20260923 22:10:10.033987 15453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:10.034304 15453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:10.035446 15453 hybrid_clock.cc:648] HybridClock initialized: now 1790201410035431 us; error 32 us; skew 500 ppm
Sep 23 22:10:10 dist-test-slave-b5jz krb5kdc[15290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:10.037835 15453 init.cc:377] Logged in from keytab as kudu/127.1.244.252@KRBTEST.COM (short username kudu)
I20260923 22:10:10.038245 15453 webserver.cc:460] Webserver started at http://127.1.244.252:46239/ using document root <none> and password file <none>
I20260923 22:10:10.038457 15453 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:10.038501 15453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:10.038640 15453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:10.039729 15453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/data/instance:
uuid: "557ebc11318b429cb8da81c97b505e1f"
format_stamp: "Formatted at 2026-09-23 22:10:10 on dist-test-slave-b5jz"
server_key: "ad679966d72f524f735dacd7807cf768"
server_key_iv: "f247ae2693ea1b85a4d303d15cea4cdd"
server_key_version: "encryptionkey@0"
I20260923 22:10:10.040072 15453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/wal/instance:
uuid: "557ebc11318b429cb8da81c97b505e1f"
format_stamp: "Formatted at 2026-09-23 22:10:10 on dist-test-slave-b5jz"
server_key: "ad679966d72f524f735dacd7807cf768"
server_key_iv: "f247ae2693ea1b85a4d303d15cea4cdd"
server_key_version: "encryptionkey@0"
I20260923 22:10:10.041349 15453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:10:10.042021 15468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:10.042197 15453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:10:10.042276 15453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/wal
uuid: "557ebc11318b429cb8da81c97b505e1f"
format_stamp: "Formatted at 2026-09-23 22:10:10 on dist-test-slave-b5jz"
server_key: "ad679966d72f524f735dacd7807cf768"
server_key_iv: "f247ae2693ea1b85a4d303d15cea4cdd"
server_key_version: "encryptionkey@0"
I20260923 22:10:10.042328 15453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:10.047461 15446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } attempt: 2
I20260923 22:10:10.047864 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.047752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60656)
0923 22:10:10.047806 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.047838 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 22:10:10.048174 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.048096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60666)
0923 22:10:10.048136 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.048158 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 22:10:10.048273 15446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43167: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43167: connect: Connection refused (error 111)
I20260923 22:10:10.065697 15453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:10.067744 15453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:10.067896 15453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:10.071779 15453 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:43167
I20260923 22:10:10.071924 15520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:43167 every 8 connection(s)
I20260923 22:10:10.072212 15453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
I20260923 22:10:10.073148 15521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:10.074851 15521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:10.076711 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 15453
I20260923 22:10:10.076838 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201256447402-2003-0/minicluster-data/master-2/wal/instance
I20260923 22:10:10.077039 2003 external_mini_cluster.cc:1428] Setting key 874db34cfd057865597786fdaa56dd42
Sep 23 22:10:10 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
Sep 23 22:10:10 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260923 22:10:10.084990 15374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.075652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36906 (local address 127.1.244.254:38437)
0923 22:10:10.075721 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.075723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.075910 (+ 187us) server_negotiation.cc:407] Connection header received
0923 22:10:10.076401 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.076403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.076415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.076472 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.077210 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.077849 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.078836 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.079066 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.079154 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.081305 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.081312 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.081312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.081319 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.083380 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.083984 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.083985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.083985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.084041 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.084309 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.084310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.084311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.084461 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.084520 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.084808 (+ 288us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.084895 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:10:10.085192 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.075636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38437 (local address 127.0.0.1:36906)
0923 22:10:10.075794 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.075830 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.076295 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.076543 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.076546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.076836 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:10.077109 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.077116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.077959 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.077960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.078746 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.078748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.079162 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.079169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.079292 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.079866 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:10.079875 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:10.081203 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:10.083491 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.083495 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.083497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.083889 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.084119 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.084120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.084120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.084233 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.084525 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:10.084532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:10.084665 (+ 133us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.084773 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.084951 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260923 22:10:10.085747 15521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:10.087132 15525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.077473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.1.244.252:43167)
0923 22:10:10.077653 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.077658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.077687 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:10:10.077759 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.077762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.077783 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.077884 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.078564 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.079347 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.080233 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.080567 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.080664 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.083461 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.083469 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.083470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.083477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.085873 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.086300 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.086302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.086303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.086364 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.086553 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.086554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.086554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.086710 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.086872 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.086943 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.087018 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
Sep 23 22:10:10 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
Sep 23 22:10:10 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:10.093497 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.086012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38479 (local address 127.0.0.1:49576)
0923 22:10:10.086128 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.086138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.086199 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.086326 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.086327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.086465 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:10.086504 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.086505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.087180 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.087181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.087943 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.087945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.088350 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.088351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.088419 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.088899 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:10.088905 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:10.090091 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:10.092382 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.092383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.092384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.092574 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.092801 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.092802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.092802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.092863 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.093162 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:10.093163 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:10.093207 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.093293 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.093393 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260923 22:10:10.093509 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.086080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.1.244.253:38479)
0923 22:10:10.086136 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.086137 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.086185 (+ 48us) server_negotiation.cc:407] Connection header received
0923 22:10:10.086262 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.086263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.086272 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.086313 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.086579 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.087072 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.088042 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.088250 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.088350 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.090188 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.090189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.090190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.090198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.092279 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.092656 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.092658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.092658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.092719 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.092942 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.092943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.092943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.093073 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.093157 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.093335 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.093414 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:10:10.094264 15521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:10.094713 15371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } attempt: 2
I20260923 22:10:10.095813 15374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.088151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36908 (local address 127.1.244.254:38437)
0923 22:10:10.088192 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.088194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.088228 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:10:10.088296 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.088297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.088306 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.088359 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.088806 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.089244 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.089938 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.090136 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.090227 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.092467 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.092468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.092468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.092475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.093967 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.094416 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.094417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.094417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.094476 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.094724 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.094725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.094725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.095120 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.095314 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.095350 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.095729 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 22:10:10 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
Sep 23 22:10:10 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
Sep 23 22:10:10 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260923 22:10:10.103536 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.094928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60674)
0923 22:10:10.095004 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.095021 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.095048 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.095357 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.095358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.095577 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:10.095613 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.095615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.096546 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.096548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.097069 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.097071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.097529 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.097530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.097584 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.098136 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:10.098140 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:10.099215 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:10.102038 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.102039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.102040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.102549 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.102820 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.102821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.102821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.102884 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.103175 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:10.103176 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:10.103222 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.103321 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.103423 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260923 22:10:10.103611 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.095030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.1.244.252:43167)
0923 22:10:10.095206 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.095208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.095215 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:10:10.095235 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.095236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.095247 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.095435 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.095691 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.096450 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.097142 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.097445 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.097709 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.099301 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.099302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.099303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.099310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.101921 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.102638 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.102639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.102639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.102706 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.102940 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.102941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.102942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.103095 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.103167 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.103349 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.103426 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":118,"threads_started":1}
I20260923 22:10:10.104220 15525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.094558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.1.244.252:43167)
0923 22:10:10.094590 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.094592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.094633 (+ 41us) server_negotiation.cc:407] Connection header received
0923 22:10:10.094860 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.094860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.094872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.094926 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.095252 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.096044 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.097108 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.097397 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.097482 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.100381 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.100383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.100383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.100391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.102199 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.102614 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.102615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.102615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.102691 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.102908 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.102909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.102909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.103092 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.103161 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.104008 (+ 847us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.104102 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:10:10.104481 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.094528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60672)
0923 22:10:10.094587 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.094597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.094650 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.094931 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.094932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.095119 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:10.095157 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.095159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.096134 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.096135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.097010 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.097014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.097479 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.097480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.097533 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.098705 (+ 1172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:10.098711 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:10.100284 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:10.102286 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.102287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.102288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.102518 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.102769 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.102770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.102770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.102842 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.103753 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:10.103753 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:10.103801 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.103884 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.104392 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 22:10:10.105037 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.096519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.1.244.253:38479)
0923 22:10:10.096554 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.096556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.096572 (+ 16us) server_negotiation.cc:407] Connection header received
0923 22:10:10.096591 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.096591 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.096600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.096647 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.097247 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.097877 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.098664 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.098977 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.099048 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.102048 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.102049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.102050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.102057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.103684 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.104082 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.104083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.104084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.104143 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.104373 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.104374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.104374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.104503 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.104669 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.104870 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.104954 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:10:10.105298 15371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06: Bootstrap starting.
I20260923 22:10:10.106056 15371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:10.106398 15371 log.cc:824] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:10.106628 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:10:10.107300 15371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06: No bootstrap required, opened a new log
I20260923 22:10:10.108016 15521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f: Bootstrap starting.
I20260923 22:10:10.108649 15521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:10.108947 15521 log.cc:824] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:10.109721 15521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f: No bootstrap required, opened a new log
I20260923 22:10:10.109769 15371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } }
I20260923 22:10:10.109894 15371 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:10.109927 15371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3188c5bd3fd64db5bd46ca4f235a7c06, State: Initialized, Role: FOLLOWER
I20260923 22:10:10.110051 15371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } }
I20260923 22:10:10.110338 15371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:10.111969 15521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } }
I20260923 22:10:10.112107 15521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:10.112143 15521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 557ebc11318b429cb8da81c97b505e1f, State: Initialized, Role: FOLLOWER
I20260923 22:10:10.112250 15521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } }
I20260923 22:10:10.112533 15548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.112914 15536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.113030 15536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:10:10.114084 15547 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260923 22:10:10.112548 15521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [sys.catalog]: configured and running, proceeding with master startup.
W20260923 22:10:10.114197 15547 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:10:10.112829 15303 master_runner.cc:186] Error getting master registration for 127.1.244.253:38479: OK, instance_id { permanent_uuid: "975263546f574b7a9c24599024caba81" instance_seqno: 1790201409906207 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:10:10.115090 15303 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:10:10.116007 15548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:10:10.116617 15560 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:10:10.116739 15560 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:10:10.118283 15525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.108330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60686 (local address 127.1.244.252:43167)
0923 22:10:10.108372 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.108375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.108771 (+ 396us) server_negotiation.cc:407] Connection header received
0923 22:10:10.108871 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.108872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.108881 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.108915 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.109757 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.110560 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.111692 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.112010 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.112054 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.113853 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.113855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.113855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.113864 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.116923 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.117333 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.117334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.117335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.117396 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.117656 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.117657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.117657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.117786 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.117867 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.118093 (+ 226us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.118188 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:10:10.117205 15453 master_runner.cc:186] Error getting master registration for 127.1.244.253:38479: OK, instance_id { permanent_uuid: "975263546f574b7a9c24599024caba81" instance_seqno: 1790201409906207 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:10:10.119381 15453 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:10:10.119634 15374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.108075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36910 (local address 127.1.244.254:38437)
0923 22:10:10.108458 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.108460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.108645 (+ 185us) server_negotiation.cc:407] Connection header received
0923 22:10:10.109414 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.109415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.109428 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.109490 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.110173 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.110910 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.112033 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.112691 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.112764 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.114748 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.114750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.114750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.114758 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.118082 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.118461 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.118462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.118462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.118608 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.118827 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.118828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.118828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.118952 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.119032 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.119404 (+ 372us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.119480 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362}
I20260923 22:10:10.123458 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.108196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.1.244.253:38479)
0923 22:10:10.108332 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.108334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.108452 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:10:10.108551 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.108552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.108563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.108615 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.115069 (+ 6454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.115845 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.118838 (+ 2993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.119159 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.119200 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.120711 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.120712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.120713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.120720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.122229 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.122610 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.122611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.122611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.122674 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.122902 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.122903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.122903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.123014 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.123129 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.123266 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.123345 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260923 22:10:10.139422 15446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } attempt: 3
Sep 23 22:10:10 dist-test-slave-b5jz krb5kdc[15290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260923 22:10:10.146118 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.139745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43167 (local address 127.0.0.1:60696)
0923 22:10:10.139795 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.139806 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.139849 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.139994 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.139995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.140152 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:10.140191 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.140193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.140815 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.140816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.141302 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.141304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.141652 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.141653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.141710 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.142163 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:10.142168 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:10.143347 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:10.145045 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.145046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.145046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.145270 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.145476 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:10.145477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:10.145477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:10.145550 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:10.145840 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:10.145841 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:10.145890 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.145953 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.146053 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:10:10.146484 15525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.139796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.1.244.252:43167)
0923 22:10:10.139832 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.139835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.139843 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:10.139907 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.139908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.139917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.139989 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.140273 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.140734 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.141389 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.141579 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.141655 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.143436 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.143438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.143439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.143445 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.144921 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.145336 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.145338 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.145338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.145403 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.145621 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.145622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.145623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.145754 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.145811 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.146327 (+ 516us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.146398 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:10:10.147863 15446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81: Bootstrap starting.
I20260923 22:10:10.148490 15446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:10.148782 15446 log.cc:824] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:10.149533 15446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81: No bootstrap required, opened a new log
I20260923 22:10:10.150911 15446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } }
I20260923 22:10:10.151062 15446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:10.151110 15446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 975263546f574b7a9c24599024caba81, State: Initialized, Role: FOLLOWER
I20260923 22:10:10.151278 15446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } }
I20260923 22:10:10.151573 15563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.151643 15446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:10.151710 15563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:10:10.153422 15574 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:10:10.153540 15574 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 22:10:10.154052 15399 tablet.cc:2369] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:10.156248 15378 master_runner.cc:263] No leader master found from master 975263546f574b7a9c24599024caba81
I20260923 22:10:10.156296 15378 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:10:10.169484 15563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:10.169588 15563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } }
I20260923 22:10:10.169800 15563 leader_election.cc:288] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3188c5bd3fd64db5bd46ca4f235a7c06 (127.1.244.254:38437), 557ebc11318b429cb8da81c97b505e1f (127.1.244.252:43167)
I20260923 22:10:10.170210 15345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "975263546f574b7a9c24599024caba81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" is_pre_election: true
I20260923 22:10:10.170372 15345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 975263546f574b7a9c24599024caba81 in term 0.
I20260923 22:10:10.170759 15495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "975263546f574b7a9c24599024caba81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "557ebc11318b429cb8da81c97b505e1f" is_pre_election: true
I20260923 22:10:10.170889 15495 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 975263546f574b7a9c24599024caba81 in term 0.
I20260923 22:10:10.171118 15396 leader_election.cc:302] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 557ebc11318b429cb8da81c97b505e1f, 975263546f574b7a9c24599024caba81; no voters:
I20260923 22:10:10.171304 15563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:10.171352 15563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:10.171378 15563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:10.172353 15563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } }
I20260923 22:10:10.172458 15563 leader_election.cc:288] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [CANDIDATE]: Term 1 election: Requested vote from peers 3188c5bd3fd64db5bd46ca4f235a7c06 (127.1.244.254:38437), 557ebc11318b429cb8da81c97b505e1f (127.1.244.252:43167)
I20260923 22:10:10.172917 15345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "975263546f574b7a9c24599024caba81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06"
I20260923 22:10:10.173005 15345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:10.173893 15345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 975263546f574b7a9c24599024caba81 in term 1.
I20260923 22:10:10.174108 15495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "975263546f574b7a9c24599024caba81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "557ebc11318b429cb8da81c97b505e1f"
I20260923 22:10:10.174201 15495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:10.175006 15495 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 975263546f574b7a9c24599024caba81 in term 1.
I20260923 22:10:10.175222 15396 leader_election.cc:302] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 557ebc11318b429cb8da81c97b505e1f, 975263546f574b7a9c24599024caba81; no voters:
I20260923 22:10:10.175354 15563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:10.175500 15563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [term 1 LEADER]: Becoming Leader. State: Replica: 975263546f574b7a9c24599024caba81, State: Running, Role: LEADER
I20260923 22:10:10.175611 15563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } }
I20260923 22:10:10.175937 15563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 975263546f574b7a9c24599024caba81. Latest consensus state: current_term: 1 leader_uuid: "975263546f574b7a9c24599024caba81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.176009 15563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:10.177126 15578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:10.177354 15578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:10.179361 15495 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [term 1 FOLLOWER]: Refusing update from remote peer 975263546f574b7a9c24599024caba81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:10.179839 15345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [term 1 FOLLOWER]: Refusing update from remote peer 975263546f574b7a9c24599024caba81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:10.180099 15563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:10.180392 15563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:10.182242 15548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 975263546f574b7a9c24599024caba81. Latest consensus state: current_term: 1 leader_uuid: "975263546f574b7a9c24599024caba81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.182322 15548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:10:10.183431 15536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 975263546f574b7a9c24599024caba81. Latest consensus state: current_term: 1 leader_uuid: "975263546f574b7a9c24599024caba81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.183537 15536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:10:10.184739 15563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "975263546f574b7a9c24599024caba81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.184813 15563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:10.185214 15536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "975263546f574b7a9c24599024caba81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.185286 15536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:10:10.185527 15579 mvcc.cc:205] Tried to move back new op lower bound from 7332664976092221440 to 7332664976079785984. Current Snapshot: MvccSnapshot[applied={T|T < 7332664976092221440}]
I20260923 22:10:10.186049 15582 mvcc.cc:205] Tried to move back new op lower bound from 7332664976092221440 to 7332664976079785984. Current Snapshot: MvccSnapshot[applied={T|T < 7332664976092221440}]
I20260923 22:10:10.188020 15578 catalog_manager.cc:1347] Generated new cluster ID: 7aeaa4ecb2754472a1f496b3b30fa51c
I20260923 22:10:10.188066 15578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:10.188241 15576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "975263546f574b7a9c24599024caba81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.188336 15576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:10.188891 15580 mvcc.cc:205] Tried to move back new op lower bound from 7332664976092221440 to 7332664976079785984. Current Snapshot: MvccSnapshot[applied={T|T < 7332664976092221440}]
I20260923 22:10:10.191632 15548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "975263546f574b7a9c24599024caba81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3188c5bd3fd64db5bd46ca4f235a7c06" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38437 } } peers { permanent_uuid: "975263546f574b7a9c24599024caba81" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38479 } } peers { permanent_uuid: "557ebc11318b429cb8da81c97b505e1f" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43167 } } }
I20260923 22:10:10.191709 15548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:10:10.203814 15578 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:10.204986 15578 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:10.233122 15578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 975263546f574b7a9c24599024caba81: Generated new TSK 0
I20260923 22:10:10.233415 15578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:10:11.114665 15547 catalog_manager.cc:1259] Loaded cluster ID: 7aeaa4ecb2754472a1f496b3b30fa51c
I20260923 22:10:11.114753 15547 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06: loading cluster ID for follower catalog manager: success
I20260923 22:10:11.115638 15547 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06: acquiring CA information for follower catalog manager: success
I20260923 22:10:11.115957 15547 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3188c5bd3fd64db5bd46ca4f235a7c06: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:10:11.117123 15560 catalog_manager.cc:1259] Loaded cluster ID: 7aeaa4ecb2754472a1f496b3b30fa51c
I20260923 22:10:11.117172 15560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f: loading cluster ID for follower catalog manager: success
I20260923 22:10:11.117900 15560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f: acquiring CA information for follower catalog manager: success
I20260923 22:10:11.118173 15560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 557ebc11318b429cb8da81c97b505e1f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:10:11.295598 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 15303
I20260923 22:10:11.300700 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 15378
I20260923 22:10:11.305361 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 15453
2026-09-23T22:10:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2830 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:10:11 dist-test-slave-b5jz krb5kdc[15595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:10:11 dist-test-slave-b5jz krb5kdc[15595](info): set up 2 sockets
Sep 23 22:10:11 dist-test-slave-b5jz krb5kdc[15595](info): commencing operation
krb5kdc: starting...
W20260923 22:10:12.343724 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:10:12 dist-test-slave-b5jz krb5kdc[15595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:10:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:10:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:10:12.456313 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42851
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:42851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:12.537533 15608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:12.537730 15608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:12.537752 15608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:12.538854 15608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:12.538892 15608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:12.538906 15608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:12.538918 15608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:12.538930 15608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:12.540396 15608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:42851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:12.540589 15608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:12.540759 15608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:10:12.545581 15608 server_base.cc:1034] running on GCE node
W20260923 22:10:12.545526 15614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:12.545538 15616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:12.545526 15613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:12.545989 15608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:12.546288 15608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:12.547457 15608 hybrid_clock.cc:648] HybridClock initialized: now 1790201412547452 us; error 36 us; skew 500 ppm
Sep 23 22:10:12 dist-test-slave-b5jz krb5kdc[15595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:12.549923 15608 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:10:12.550436 15608 webserver.cc:460] Webserver started at http://127.1.244.254:37273/ using document root <none> and password file <none>
I20260923 22:10:12.550678 15608 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:12.550747 15608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:12.550894 15608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:12.552021 15608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "c2a592ab1f454dd5b8285413ea369d75"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-b5jz"
server_key: "397d51e3f2dc9bc8ad03fde5820193f7"
server_key_iv: "62971dae031ddaf90986b6eb3b1e98a8"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.552380 15608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "c2a592ab1f454dd5b8285413ea369d75"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-b5jz"
server_key: "397d51e3f2dc9bc8ad03fde5820193f7"
server_key_iv: "62971dae031ddaf90986b6eb3b1e98a8"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.553712 15608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:10:12.554505 15623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:12.554697 15608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:12.554791 15608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "c2a592ab1f454dd5b8285413ea369d75"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-b5jz"
server_key: "397d51e3f2dc9bc8ad03fde5820193f7"
server_key_iv: "62971dae031ddaf90986b6eb3b1e98a8"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.554874 15608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:12.602931 15608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:12.605126 15608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:12.605299 15608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:12.609372 15608 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:42851
I20260923 22:10:12.609411 15675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:42851 every 8 connection(s)
I20260923 22:10:12.609809 15608 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:12.610677 15676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:12.611054 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 15608
I20260923 22:10:12.611173 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:10:12.611363 2003 external_mini_cluster.cc:1428] Setting key 13577bc9d8f6b1e28729d7cfa82bb9dd
I20260923 22:10:12.613813 15676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75: Bootstrap starting.
I20260923 22:10:12.614503 15676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:12.614789 15676 log.cc:824] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:12.615594 15676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75: No bootstrap required, opened a new log
I20260923 22:10:12.617965 15676 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2a592ab1f454dd5b8285413ea369d75" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42851 } }
I20260923 22:10:12.618155 15676 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:12.618189 15676 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2a592ab1f454dd5b8285413ea369d75, State: Initialized, Role: FOLLOWER
I20260923 22:10:12.618345 15676 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2a592ab1f454dd5b8285413ea369d75" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42851 } }
I20260923 22:10:12.618431 15676 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:12.618469 15676 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:12.618516 15676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [term 0 FOLLOWER]: Advancing to term 1
Sep 23 22:10:12 dist-test-slave-b5jz krb5kdc[15595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:12.619148 15676 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2a592ab1f454dd5b8285413ea369d75" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42851 } }
I20260923 22:10:12.619257 15676 leader_election.cc:302] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2a592ab1f454dd5b8285413ea369d75; no voters:
I20260923 22:10:12.619421 15676 leader_election.cc:288] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:10:12.619463 15681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:12.619625 15681 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [term 1 LEADER]: Becoming Leader. State: Replica: c2a592ab1f454dd5b8285413ea369d75, State: Running, Role: LEADER
I20260923 22:10:12.619733 15676 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:12.619789 15681 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2a592ab1f454dd5b8285413ea369d75" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42851 } }
I20260923 22:10:12.620112 15681 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2a592ab1f454dd5b8285413ea369d75. Latest consensus state: current_term: 1 leader_uuid: "c2a592ab1f454dd5b8285413ea369d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2a592ab1f454dd5b8285413ea369d75" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42851 } } }
I20260923 22:10:12.620256 15681 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:12.620496 15689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:12.621034 15689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:12.620148 15683 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2a592ab1f454dd5b8285413ea369d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2a592ab1f454dd5b8285413ea369d75" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42851 } } }
I20260923 22:10:12.621512 15683 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:12.622699 15689 catalog_manager.cc:1347] Generated new cluster ID: ed5e110e45624f48915043112f144218
I20260923 22:10:12.622746 15689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:12.622938 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.611886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41950 (local address 127.1.244.254:42851)
0923 22:10:12.612083 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.612093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.612129 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:10:12.612663 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.612672 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.612732 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.612880 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.613484 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.614842 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.615937 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.616559 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.616666 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.619102 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.619106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.619107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.619118 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.621280 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.621694 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.621698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.621698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.621816 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.622026 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.622027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.622027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.622350 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.622519 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.622638 (+ 119us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.622772 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260923 22:10:12.623845 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:10:12.630515 15689 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:12.631111 15689 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:12.640066 15689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2a592ab1f454dd5b8285413ea369d75: Generated new TSK 0
I20260923 22:10:12.640268 15689 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:10:12.677932 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 15608
2026-09-23T22:10:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1371 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:10:12 dist-test-slave-b5jz krb5kdc[15703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:10:12 dist-test-slave-b5jz krb5kdc[15703](info): set up 2 sockets
Sep 23 22:10:12 dist-test-slave-b5jz krb5kdc[15703](info): commencing operation
krb5kdc: starting...
W20260923 22:10:13.727638 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:10:13 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:10:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:10:13.844221 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:43429
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:43429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:13.923704 15716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:13.923898 15716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:13.923919 15716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:13.924997 15716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:13.925030 15716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:13.925045 15716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:13.925057 15716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:13.925069 15716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:13.926460 15716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:43429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:43429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:13.926640 15716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:13.926797 15716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:13.931504 15721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:13.931522 15724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:13.931504 15722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:13.931640 15716 server_base.cc:1034] running on GCE node
I20260923 22:10:13.932032 15716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:13.932292 15716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:13.933459 15716 hybrid_clock.cc:648] HybridClock initialized: now 1790201413933458 us; error 41 us; skew 500 ppm
Sep 23 22:10:13 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201413, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:13.935951 15716 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:10:13.936393 15716 webserver.cc:460] Webserver started at http://127.1.244.254:32791/ using document root <none> and password file <none>
I20260923 22:10:13.936640 15716 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:13.936710 15716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:13.936856 15716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:13.937891 15716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "3974cce730fa4449ac372e4768750bb9"
format_stamp: "Formatted at 2026-09-23 22:10:13 on dist-test-slave-b5jz"
server_key: "bcb312dc48debf28415f3feeb4ec9b5e"
server_key_iv: "0306c5b17aa2a2095b678cbb5ffce1a8"
server_key_version: "encryptionkey@0"
I20260923 22:10:13.938273 15716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "3974cce730fa4449ac372e4768750bb9"
format_stamp: "Formatted at 2026-09-23 22:10:13 on dist-test-slave-b5jz"
server_key: "bcb312dc48debf28415f3feeb4ec9b5e"
server_key_iv: "0306c5b17aa2a2095b678cbb5ffce1a8"
server_key_version: "encryptionkey@0"
I20260923 22:10:13.939625 15716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:10:13.940389 15731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:13.940572 15716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:13.940675 15716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "3974cce730fa4449ac372e4768750bb9"
format_stamp: "Formatted at 2026-09-23 22:10:13 on dist-test-slave-b5jz"
server_key: "bcb312dc48debf28415f3feeb4ec9b5e"
server_key_iv: "0306c5b17aa2a2095b678cbb5ffce1a8"
server_key_version: "encryptionkey@0"
I20260923 22:10:13.940765 15716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:13.957863 15716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:13.959769 15716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:13.959932 15716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:13.963714 15716 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:43429
I20260923 22:10:13.963824 15783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:43429 every 8 connection(s)
I20260923 22:10:13.964151 15716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:13.965170 15784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:13.967506 15784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Bootstrap starting.
I20260923 22:10:13.968132 15784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:13.968382 15784 log.cc:824] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:13.968457 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 15716
I20260923 22:10:13.968598 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:10:13.968784 2003 external_mini_cluster.cc:1428] Setting key 969938f662f495026b7515c49ec6b174
I20260923 22:10:13.969254 15784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: No bootstrap required, opened a new log
I20260923 22:10:13.971596 15784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:13.971802 15784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:13.971848 15784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3974cce730fa4449ac372e4768750bb9, State: Initialized, Role: FOLLOWER
I20260923 22:10:13.972002 15784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:13.972084 15784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:13.972116 15784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:13.972163 15784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:13.972884 15784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:13.972997 15784 leader_election.cc:302] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3974cce730fa4449ac372e4768750bb9; no voters:
I20260923 22:10:13.973168 15784 leader_election.cc:288] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:10:13.973222 15789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:13.973402 15784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:13.973402 15789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 1 LEADER]: Becoming Leader. State: Replica: 3974cce730fa4449ac372e4768750bb9, State: Running, Role: LEADER
I20260923 22:10:13.973650 15789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:13.974716 15790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3974cce730fa4449ac372e4768750bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } } }
I20260923 22:10:13.974792 15791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3974cce730fa4449ac372e4768750bb9. Latest consensus state: current_term: 1 leader_uuid: "3974cce730fa4449ac372e4768750bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } } }
I20260923 22:10:13.974849 15790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:13.974910 15791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: This master's current role is: LEADER
W20260923 22:10:13.975524 15805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:10:13.975633 15805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:10:13.975695 15804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:13.975849 15804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:10:13 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:13.977641 15804 catalog_manager.cc:1347] Generated new cluster ID: 7c973244d71e472c9188f234fe8ad6ac
I20260923 22:10:13.977694 15804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:13.981216 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:13.969316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34050 (local address 127.1.244.254:43429)
0923 22:10:13.969545 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:13.969555 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:13.969594 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:10:13.970675 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:13.970685 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:13.970752 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:13.970924 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:13.971522 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:13.973389 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:13.974477 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:13.975138 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:13.975247 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:13.977921 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:13.977926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:13.977926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:13.977938 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:13.979712 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:13.980048 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:13.980051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:13.980051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:13.980150 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:13.980359 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:13.980360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:13.980361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:13.980594 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:13.980775 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:13.980869 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 22:10:13.981004 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
I20260923 22:10:14.001303 15804 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:14.001819 15804 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:14.007648 15804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Generated new TSK 0
I20260923 22:10:14.007822 15804 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:10:14.036568 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:43429
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:14.140637 15810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:14.140877 15810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:14.140916 15810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:14.142072 15810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:14.142141 15810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:14.142189 15810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:10:14.143877 15810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:14.144137 15810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:14.144366 15810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:14.150305 15818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.150308 15816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.150328 15815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:14.150767 15810 server_base.cc:1034] running on GCE node
I20260923 22:10:14.151011 15810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:14.151352 15810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:14.152535 15810 hybrid_clock.cc:648] HybridClock initialized: now 1790201414152526 us; error 31 us; skew 500 ppm
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:14.155687 15810 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:10:14.156168 15810 webserver.cc:460] Webserver started at http://127.1.244.193:42673/ using document root <none> and password file <none>
I20260923 22:10:14.156441 15810 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:14.156494 15810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:14.156659 15810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:14.157783 15810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "a974b8fb65114f9889164c24543f8d76"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "24bf8451f0882c1461d01ae569b211a4"
server_key_iv: "9a6638bde24fba672a1e8158bf32e7af"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.158183 15810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "a974b8fb65114f9889164c24543f8d76"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "24bf8451f0882c1461d01ae569b211a4"
server_key_iv: "9a6638bde24fba672a1e8158bf32e7af"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.159621 15810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:10:14.160377 15825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.160558 15810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:14.160666 15810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "a974b8fb65114f9889164c24543f8d76"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "24bf8451f0882c1461d01ae569b211a4"
server_key_iv: "9a6638bde24fba672a1e8158bf32e7af"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.160756 15810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:14.180545 15810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:14.182480 15810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:14.182653 15810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:14.182891 15810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:14.183224 15810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:14.183283 15810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.183327 15810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:14.183362 15810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.189502 15810 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:36263
I20260923 22:10:14.189570 15938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:36263 every 8 connection(s)
I20260923 22:10:14.189919 15810 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:14.192414 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 15810
I20260923 22:10:14.192550 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:10:14.192734 2003 external_mini_cluster.cc:1428] Setting key 0e95ae7bdaa2063e4bfa30cf43983b8e
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:14.199378 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.190660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53435 (local address 127.1.244.254:43429)
0923 22:10:14.190717 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.190719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.190923 (+ 204us) server_negotiation.cc:407] Connection header received
0923 22:10:14.191457 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.191459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.191471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.191529 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:14.192470 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.193088 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.194093 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.194326 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.194405 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.196493 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:14.196495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:14.196495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:14.196504 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:14.197983 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.198396 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.198397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.198397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.198460 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.198723 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.198724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.198724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.198841 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:14.198911 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.199189 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.199285 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:10:14.199549 15941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.190635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:53435)
0923 22:10:14.190799 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.190833 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.191336 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.191617 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.191620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.192182 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:14.192364 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.192368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.193193 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.193194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.193993 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.193995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.194413 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.194414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.194537 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.195057 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:14.195065 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:14.196396 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:14.198085 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.198092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.198094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.198318 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.198530 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.198531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.198531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.198646 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.198914 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:14.198920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:14.199056 (+ 136us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.199167 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.199320 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":103,"threads_started":1}
I20260923 22:10:14.200062 15939 heartbeater.cc:347] Connected to a master server at 127.1.244.254:43429
I20260923 22:10:14.200156 15939 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260923 22:10:14.200344 15939 heartbeater.cc:511] Master 127.1.244.254:43429 requested a full tablet report, sending...
I20260923 22:10:14.200860 15748 ts_manager.cc:194] Registered new tserver with Master: a974b8fb65114f9889164c24543f8d76 (127.1.244.193:36263)
I20260923 22:10:14.201653 15748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:53435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:10:14.215966 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:43429
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:14.295621 15944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:14.295823 15944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:14.295846 15944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:14.297009 15944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:14.297051 15944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:14.297077 15944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:10:14.298527 15944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:14.298781 15944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:14.299014 15944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:14.303908 15949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.303902 15950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.303879 15952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:14.304282 15944 server_base.cc:1034] running on GCE node
I20260923 22:10:14.304512 15944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:14.304760 15944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:14.305923 15944 hybrid_clock.cc:648] HybridClock initialized: now 1790201414305908 us; error 32 us; skew 500 ppm
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:14.308470 15944 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:10:14.308903 15944 webserver.cc:460] Webserver started at http://127.1.244.194:33275/ using document root <none> and password file <none>
I20260923 22:10:14.309142 15944 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:14.309211 15944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:14.309362 15944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:14.310423 15944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "711362bc8ff54e03922d2fac51a3ffe0"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "59243b5e4883f635826aabcf0bcf4f50"
server_key_iv: "417d686676e63c44608dc24dd11de57a"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.310791 15944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "711362bc8ff54e03922d2fac51a3ffe0"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "59243b5e4883f635826aabcf0bcf4f50"
server_key_iv: "417d686676e63c44608dc24dd11de57a"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.312156 15944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:10:14.312867 15959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.313031 15944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:14.313136 15944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "711362bc8ff54e03922d2fac51a3ffe0"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "59243b5e4883f635826aabcf0bcf4f50"
server_key_iv: "417d686676e63c44608dc24dd11de57a"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.313208 15944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:14.324146 15944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:14.326372 15944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:14.326534 15944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:14.326766 15944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:14.327055 15944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:14.327107 15944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.327148 15944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:14.327176 15944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.332931 15944 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:37601
I20260923 22:10:14.333024 16072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:37601 every 8 connection(s)
I20260923 22:10:14.333307 15944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:14.340389 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 15944
I20260923 22:10:14.340508 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:10:14.340715 2003 external_mini_cluster.cc:1428] Setting key 730e117462a9dc1fa84081e521e5657a
I20260923 22:10:14.342341 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.334002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:38527 (local address 127.1.244.254:43429)
0923 22:10:14.334058 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.334060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.334255 (+ 195us) server_negotiation.cc:407] Connection header received
0923 22:10:14.334698 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.334699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.334708 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.334763 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:14.335705 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.336209 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.337139 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.337323 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.337371 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.339281 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:14.339282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:14.339282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:14.339289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:14.340909 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.341372 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.341372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.341373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.341441 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.341736 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.341737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.341737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.341841 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:14.341892 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.342217 (+ 325us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.342279 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:10:14.342670 16075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.333952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:38527)
0923 22:10:14.334113 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.334165 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.334614 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.334815 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.334818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.335366 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:14.335602 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.335608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.336286 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.336287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.337058 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.337059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.337389 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.337390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.337486 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.337893 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:14.337901 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:14.339202 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:14.341012 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.341019 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.341021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.341285 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.341518 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.341518 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.341519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.341671 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.341908 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:14.341915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:14.342083 (+ 168us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.342199 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.342370 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260923 22:10:14.343183 16073 heartbeater.cc:347] Connected to a master server at 127.1.244.254:43429
I20260923 22:10:14.343297 16073 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:14.343464 16073 heartbeater.cc:511] Master 127.1.244.254:43429 requested a full tablet report, sending...
I20260923 22:10:14.343886 15748 ts_manager.cc:194] Registered new tserver with Master: 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601)
I20260923 22:10:14.344344 15748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:38527
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:10:14.363863 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:43429
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:14.444432 16078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:14.444643 16078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:14.444666 16078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:14.445871 16078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:14.445914 16078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:14.445940 16078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:10:14.447301 16078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:14.447487 16078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:14.447772 16078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:14.452519 16084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.452607 16086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.452519 16083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:14.452745 16078 server_base.cc:1034] running on GCE node
I20260923 22:10:14.453007 16078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:14.453263 16078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:14.454428 16078 hybrid_clock.cc:648] HybridClock initialized: now 1790201414454424 us; error 35 us; skew 500 ppm
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:14.457094 16078 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:10:14.457573 16078 webserver.cc:460] Webserver started at http://127.1.244.195:38955/ using document root <none> and password file <none>
I20260923 22:10:14.457820 16078 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:14.457888 16078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:14.458043 16078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:14.459053 16078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "61f8947b4bab481494a1bcefda29cec3"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "635cd07c6dc1d859202610d857cb6e75"
server_key_iv: "27e0043aac6e22d39861aab373f6fee7"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.459407 16078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "61f8947b4bab481494a1bcefda29cec3"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "635cd07c6dc1d859202610d857cb6e75"
server_key_iv: "27e0043aac6e22d39861aab373f6fee7"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.460736 16078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:14.461458 16093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.461666 16078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.461784 16078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "61f8947b4bab481494a1bcefda29cec3"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "635cd07c6dc1d859202610d857cb6e75"
server_key_iv: "27e0043aac6e22d39861aab373f6fee7"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.461875 16078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:14.478246 16078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:14.480729 16078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:14.480912 16078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:14.481181 16078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:14.481520 16078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:14.481578 16078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.481617 16078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:14.481668 16078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.487879 16078 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:39763
I20260923 22:10:14.487929 16206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:39763 every 8 connection(s)
I20260923 22:10:14.488271 16078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:14.497198 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.489041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:39063 (local address 127.1.244.254:43429)
0923 22:10:14.489098 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.489099 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.489318 (+ 219us) server_negotiation.cc:407] Connection header received
0923 22:10:14.489762 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.489763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.489774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.489840 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:14.490739 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.491235 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.492257 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.492457 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.492526 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.494533 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:14.494534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:14.494534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:14.494544 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:14.496031 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.496388 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.496389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.496389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.496446 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.496658 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.496659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.496659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.496763 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:14.496816 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.497078 (+ 262us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.497136 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:10:14.497372 16209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.489026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:39063)
0923 22:10:14.489178 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.489214 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.489673 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.489895 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.489899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.490460 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:14.490651 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.490656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.491313 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.491313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.492180 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.492182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.492539 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.492539 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.492639 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.493097 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:14.493105 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:14.494425 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:14.496098 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.496104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.496105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.496319 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.496508 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.496509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.496509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.496600 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.496820 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:14.496824 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:14.496958 (+ 134us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.497050 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.497174 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260923 22:10:14.497798 16207 heartbeater.cc:347] Connected to a master server at 127.1.244.254:43429
I20260923 22:10:14.497886 16207 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:14.498013 16207 heartbeater.cc:511] Master 127.1.244.254:43429 requested a full tablet report, sending...
I20260923 22:10:14.498349 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 16078
I20260923 22:10:14.498386 15748 ts_manager.cc:194] Registered new tserver with Master: 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763)
I20260923 22:10:14.498464 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:10:14.498626 2003 external_mini_cluster.cc:1428] Setting key 4976fa5647ebf2730a0c3af27de1445f
I20260923 22:10:14.498811 15748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:39063
I20260923 22:10:14.499596 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:14.506310 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.500885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34064 (local address 127.1.244.254:43429)
0923 22:10:14.500948 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.500950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.501026 (+ 76us) server_negotiation.cc:407] Connection header received
0923 22:10:14.501061 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.501062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.501070 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.501119 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:14.501596 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.502118 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.502782 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.502951 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.502998 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.504194 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:14.504195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:14.504195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:14.504200 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:14.505422 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.505725 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.505726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.505727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.505773 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.505898 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.505899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.505899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.506017 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:14.506090 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.506173 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.506227 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:10:14.508251 15747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:14.509831 15747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:14.516108 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.512361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39763 (local address 127.0.0.1:54856)
0923 22:10:14.512471 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.512479 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.512518 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.512828 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.512829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.512831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:14.512881 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.512882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.513767 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.513768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.514974 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.514977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.515819 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.515819 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.515898 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.515913 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.515985 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.516032 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":75,"threads_started":1}
I20260923 22:10:14.516141 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.512438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54856 (local address 127.1.244.195:39763)
0923 22:10:14.512586 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.512590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.512598 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:14.512641 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.512644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.512658 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.512727 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:14.512967 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.513663 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.515045 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.515736 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.515809 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.515904 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.515997 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.516043 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260923 22:10:14.516343 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.511976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36263 (local address 127.0.0.1:58356)
0923 22:10:14.512076 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.512116 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.512177 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.512750 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.512751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.512752 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:14.512807 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.512809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.514140 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.514141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.515152 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.515154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.516127 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.516128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.516209 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.516213 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.516247 (+ 34us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.516284 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":1,"thread_start_us":57,"threads_started":1}
I20260923 22:10:14.516626 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.512084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.1.244.193:36263)
0923 22:10:14.512565 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.512569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.512579 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:14.512616 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.512618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.512638 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.512888 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:14.512971 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.514028 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.515229 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.516034 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.516395 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.516441 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.516484 (+ 43us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.516527 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":104,"threads_started":1}
I20260923 22:10:14.517319 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.512181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37601 (local address 127.0.0.1:37512)
0923 22:10:14.512289 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.512300 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.512357 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.512729 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.512733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.512735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:14.512807 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.512809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.515217 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.515218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.516086 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.516088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.517083 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.517084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.517148 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.517152 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.517214 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.517258 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":5,"thread_start_us":66,"threads_started":1}
I20260923 22:10:14.517241 16141 tablet_service.cc:1467] Processing CreateTablet for tablet d4f8f52fdb764b34ad31abd367e728f0 (DEFAULT_TABLE table=test-table [id=8ea619aec55b4c248d572750a72d310f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:14.517369 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.512301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.1.244.194:37601)
0923 22:10:14.512536 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.512540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.512554 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:10:14.512598 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.512600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.512617 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.512710 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:14.512898 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.514039 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.516176 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.516996 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.517066 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.517119 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.517232 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.517282 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":94,"threads_started":1}
I20260923 22:10:14.517331 15873 tablet_service.cc:1467] Processing CreateTablet for tablet d4f8f52fdb764b34ad31abd367e728f0 (DEFAULT_TABLE table=test-table [id=8ea619aec55b4c248d572750a72d310f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:14.517596 16141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4f8f52fdb764b34ad31abd367e728f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:14.517663 15873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4f8f52fdb764b34ad31abd367e728f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:14.518090 16007 tablet_service.cc:1467] Processing CreateTablet for tablet d4f8f52fdb764b34ad31abd367e728f0 (DEFAULT_TABLE table=test-table [id=8ea619aec55b4c248d572750a72d310f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:14.518392 16007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4f8f52fdb764b34ad31abd367e728f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:14.520517 16224 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: Bootstrap starting.
I20260923 22:10:14.521114 16224 tablet_bootstrap.cc:654] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:14.521282 16225 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Bootstrap starting.
I20260923 22:10:14.521611 16226 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Bootstrap starting.
I20260923 22:10:14.521770 16224 log.cc:824] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:14.522017 16225 tablet_bootstrap.cc:654] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:14.522332 16226 tablet_bootstrap.cc:654] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:14.522626 16225 log.cc:824] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:14.522881 16226 log.cc:824] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:14.522925 16224 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: No bootstrap required, opened a new log
I20260923 22:10:14.523005 16224 ts_tablet_manager.cc:1397] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:10:14.523365 16225 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: No bootstrap required, opened a new log
I20260923 22:10:14.523437 16225 ts_tablet_manager.cc:1397] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:14.523576 16226 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: No bootstrap required, opened a new log
I20260923 22:10:14.523643 16226 ts_tablet_manager.cc:1397] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:14.524930 16224 raft_consensus.cc:348] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.524940 16226 raft_consensus.cc:348] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.525106 16224 raft_consensus.cc:374] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:14.525131 16226 raft_consensus.cc:374] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:14.525146 16224 raft_consensus.cc:729] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f8947b4bab481494a1bcefda29cec3, State: Initialized, Role: FOLLOWER
I20260923 22:10:14.525202 16226 raft_consensus.cc:729] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711362bc8ff54e03922d2fac51a3ffe0, State: Initialized, Role: FOLLOWER
I20260923 22:10:14.525262 16224 consensus_queue.cc:260] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.525362 16226 consensus_queue.cc:260] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.525571 16224 ts_tablet_manager.cc:1428] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:10:14.525525 16225 raft_consensus.cc:348] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.525645 16226 ts_tablet_manager.cc:1428] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:14.525676 16073 heartbeater.cc:503] Master 127.1.244.254:43429 was elected leader, sending a full tablet report...
I20260923 22:10:14.525731 16207 heartbeater.cc:503] Master 127.1.244.254:43429 was elected leader, sending a full tablet report...
I20260923 22:10:14.525743 16225 raft_consensus.cc:374] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:14.525792 16225 raft_consensus.cc:729] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a974b8fb65114f9889164c24543f8d76, State: Initialized, Role: FOLLOWER
I20260923 22:10:14.525900 16225 consensus_queue.cc:260] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.526239 15939 heartbeater.cc:503] Master 127.1.244.254:43429 was elected leader, sending a full tablet report...
I20260923 22:10:14.526435 16225 ts_tablet_manager.cc:1428] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:10:14.546525 16230 raft_consensus.cc:482] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:14.546622 16230 raft_consensus.cc:504] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.547078 16230 leader_election.cc:288] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a974b8fb65114f9889164c24543f8d76 (127.1.244.193:36263), 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601)
I20260923 22:10:14.550213 16209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.547239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36263 (local address 127.1.244.195:42917)
0923 22:10:14.547334 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.547345 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.547430 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.547585 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.547586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.547587 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:14.547666 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.547667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.548410 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.548411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.549329 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.549331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.549901 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.549902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.550007 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.550012 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.550070 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.550115 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":27}
I20260923 22:10:14.550355 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.547456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:55255 (local address 127.1.244.194:37601)
0923 22:10:14.547500 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.547502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.547739 (+ 237us) server_negotiation.cc:407] Connection header received
0923 22:10:14.547778 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.547779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.547789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.547855 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:14.547976 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.548560 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.549509 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.549972 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.550054 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.550063 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.550233 (+ 170us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.550278 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:10:14.550338 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.547367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37601 (local address 127.1.244.195:55255)
0923 22:10:14.547661 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.547670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.547731 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.547861 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.547861 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.547863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:14.547925 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.547927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.548647 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.548648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.549405 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.549408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.550070 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.550071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.550148 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.550152 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.550196 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.550225 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":99,"threads_started":1}
I20260923 22:10:14.550526 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.547288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:42917 (local address 127.1.244.193:36263)
0923 22:10:14.547346 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.547348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.547411 (+ 63us) server_negotiation.cc:407] Connection header received
0923 22:10:14.547493 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.547494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.547503 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.547580 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:14.547758 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.548328 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.549416 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.549819 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.549874 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.549881 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.550394 (+ 513us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.550447 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:10:14.550787 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f8f52fdb764b34ad31abd367e728f0" candidate_uuid: "61f8947b4bab481494a1bcefda29cec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "711362bc8ff54e03922d2fac51a3ffe0" is_pre_election: true
I20260923 22:10:14.550818 15893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f8f52fdb764b34ad31abd367e728f0" candidate_uuid: "61f8947b4bab481494a1bcefda29cec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a974b8fb65114f9889164c24543f8d76" is_pre_election: true
I20260923 22:10:14.550936 16027 raft_consensus.cc:2454] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61f8947b4bab481494a1bcefda29cec3 in term 0.
I20260923 22:10:14.550940 15893 raft_consensus.cc:2454] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61f8947b4bab481494a1bcefda29cec3 in term 0.
I20260923 22:10:14.551160 16097 leader_election.cc:302] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61f8947b4bab481494a1bcefda29cec3, a974b8fb65114f9889164c24543f8d76; no voters:
I20260923 22:10:14.551308 16230 raft_consensus.cc:2781] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:14.551378 16230 raft_consensus.cc:482] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:14.551397 16230 raft_consensus.cc:3037] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:14.552201 16230 raft_consensus.cc:504] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.552340 16230 leader_election.cc:288] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [CANDIDATE]: Term 1 election: Requested vote from peers a974b8fb65114f9889164c24543f8d76 (127.1.244.193:36263), 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601)
I20260923 22:10:14.552526 15893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f8f52fdb764b34ad31abd367e728f0" candidate_uuid: "61f8947b4bab481494a1bcefda29cec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a974b8fb65114f9889164c24543f8d76"
I20260923 22:10:14.552536 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f8f52fdb764b34ad31abd367e728f0" candidate_uuid: "61f8947b4bab481494a1bcefda29cec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "711362bc8ff54e03922d2fac51a3ffe0"
I20260923 22:10:14.552601 15893 raft_consensus.cc:3037] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:14.552632 16027 raft_consensus.cc:3037] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:14.553323 16027 raft_consensus.cc:2454] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61f8947b4bab481494a1bcefda29cec3 in term 1.
I20260923 22:10:14.553324 15893 raft_consensus.cc:2454] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61f8947b4bab481494a1bcefda29cec3 in term 1.
I20260923 22:10:14.553504 16097 leader_election.cc:302] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61f8947b4bab481494a1bcefda29cec3, 711362bc8ff54e03922d2fac51a3ffe0; no voters:
I20260923 22:10:14.553635 16230 raft_consensus.cc:2781] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:14.553835 16230 raft_consensus.cc:686] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 1 LEADER]: Becoming Leader. State: Replica: 61f8947b4bab481494a1bcefda29cec3, State: Running, Role: LEADER
I20260923 22:10:14.554082 16230 consensus_queue.cc:237] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.554924 15747 catalog_manager.cc:5495] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "61f8947b4bab481494a1bcefda29cec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } health_report { overall_health: HEALTHY } } }
I20260923 22:10:14.564206 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.561886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54858 (local address 127.1.244.195:39763)
0923 22:10:14.561944 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.561946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.561956 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:14.561992 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.561993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.562008 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.562070 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:14.562189 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.562681 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.563485 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.563733 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.563799 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.563926 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:14.564021 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:14.564089 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.564124 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.564141 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:10:14.566056 16027 raft_consensus.cc:1261] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Refusing update from remote peer 61f8947b4bab481494a1bcefda29cec3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:14.566126 15893 raft_consensus.cc:1261] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Refusing update from remote peer 61f8947b4bab481494a1bcefda29cec3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:14.566404 16230 consensus_queue.cc:1035] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:14.566541 16234 consensus_queue.cc:1035] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:14.569042 16239 mvcc.cc:205] Tried to move back new op lower bound from 7332664994060029952 to 7332664994014294016. Current Snapshot: MvccSnapshot[applied={T|T < 7332664994060029952}]
I20260923 22:10:14.569684 16237 mvcc.cc:205] Tried to move back new op lower bound from 7332664994060029952 to 7332664994014294016. Current Snapshot: MvccSnapshot[applied={T|T < 7332664994060029952 or (T in {7332664994060029952})}]
I20260923 22:10:14.570926 15747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:14.571020 15747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:14.571810 15747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Sending DeleteTablet for 3 replicas of tablet d4f8f52fdb764b34ad31abd367e728f0
I20260923 22:10:14.572129 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 15810
I20260923 22:10:14.572228 15873 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f8f52fdb764b34ad31abd367e728f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:14 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:58356
I20260923 22:10:14.572301 16007 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f8f52fdb764b34ad31abd367e728f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:14 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:37512
I20260923 22:10:14.572502 16141 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f8f52fdb764b34ad31abd367e728f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:14 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:54856
I20260923 22:10:14.572688 16248 tablet_replica.cc:330] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: stopping tablet replica
I20260923 22:10:14.572772 16248 raft_consensus.cc:2229] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:10:14.572875 16248 raft_consensus.cc:2258] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:14.573311 16248 ts_tablet_manager.cc:1905] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:14.575162 16248 ts_tablet_manager.cc:1918] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:14.575240 16248 log.cc:1196] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/d4f8f52fdb764b34ad31abd367e728f0
I20260923 22:10:14.575510 16248 ts_tablet_manager.cc:1939] T d4f8f52fdb764b34ad31abd367e728f0 P 61f8947b4bab481494a1bcefda29cec3: Deleting consensus metadata
I20260923 22:10:14.576037 15735 catalog_manager.cc:4841] TS 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763): tablet d4f8f52fdb764b34ad31abd367e728f0 (table test-table [id=8ea619aec55b4c248d572750a72d310f]) successfully deleted
W20260923 22:10:14.578276 15735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:36263 (error 108)
I20260923 22:10:14.578397 16247 tablet_replica.cc:330] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: stopping tablet replica
I20260923 22:10:14.578511 16247 raft_consensus.cc:2229] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:14.578637 16247 raft_consensus.cc:2258] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:14.578881 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 15944
I20260923 22:10:14.578896 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.578755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36263 (local address 127.0.0.1:58368)
0923 22:10:14.578824 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.578863 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260923 22:10:14.579101 16247 ts_tablet_manager.cc:1905] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:10:14.579668 15735 catalog_manager.cc:4584] TS a974b8fb65114f9889164c24543f8d76 (127.1.244.193:36263): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4f8f52fdb764b34ad31abd367e728f0: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36263: connect: Connection refused (error 111)
I20260923 22:10:14.586721 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 16078
I20260923 22:10:14.586757 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.586652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37601 (local address 127.0.0.1:37526)
0923 22:10:14.586695 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.586726 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.194:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 22:10:14.591308 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 15716
I20260923 22:10:14.596477 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:43429
--webserver_interface=127.1.244.254
--webserver_port=32791
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:43429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:14.676915 16250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:14.677238 16250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:14.677295 16250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:14.678632 16250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:14.678738 16250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:14.678764 16250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:14.678787 16250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:14.678808 16250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:14.680593 16250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:43429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:43429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=32791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:14.680954 16250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:14.681228 16250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:14.690562 16257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.690567 16256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.690569 16259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:14.690840 16250 server_base.cc:1034] running on GCE node
I20260923 22:10:14.691138 16250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:14.691521 16250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:14.692759 16250 hybrid_clock.cc:648] HybridClock initialized: now 1790201414692743 us; error 30 us; skew 500 ppm
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:14.695504 16250 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:10:14.696005 16250 webserver.cc:460] Webserver started at http://127.1.244.254:32791/ using document root <none> and password file <none>
I20260923 22:10:14.696259 16250 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:14.696316 16250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:14.697796 16250 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:14.698453 16266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.698608 16250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:14.698757 16250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "3974cce730fa4449ac372e4768750bb9"
format_stamp: "Formatted at 2026-09-23 22:10:13 on dist-test-slave-b5jz"
server_key: "bcb312dc48debf28415f3feeb4ec9b5e"
server_key_iv: "0306c5b17aa2a2095b678cbb5ffce1a8"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.699195 16250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:14.707753 16250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:14.709700 16250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:14.709868 16250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:14.713994 16250 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:43429
I20260923 22:10:14.714039 16318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:43429 every 8 connection(s)
I20260923 22:10:14.714439 16250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:14.716109 16319 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:10:14.716781 16319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Bootstrap starting.
I20260923 22:10:14.721076 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 16250
I20260923 22:10:14.721362 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:36263
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:43429
--webserver_port=42673
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:14.725172 16319 log.cc:824] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:14.726969 16319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:14.727250 16319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Bootstrap complete.
I20260923 22:10:14.729465 16319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:14.729707 16319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3974cce730fa4449ac372e4768750bb9, State: Initialized, Role: FOLLOWER
I20260923 22:10:14.729902 16319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:14.729996 16319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:14.730054 16319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:14.730119 16319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:10:14.731127 16319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:14.731247 16319 leader_election.cc:302] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3974cce730fa4449ac372e4768750bb9; no voters:
I20260923 22:10:14.731465 16319 leader_election.cc:288] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:10:14.731495 16323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:10:14.731635 16323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 2 LEADER]: Becoming Leader. State: Replica: 3974cce730fa4449ac372e4768750bb9, State: Running, Role: LEADER
I20260923 22:10:14.731731 16323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:14.732052 16325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3974cce730fa4449ac372e4768750bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } } }
I20260923 22:10:14.732151 16325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:14.731750 16319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:14.732061 16324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3974cce730fa4449ac372e4768750bb9. Latest consensus state: current_term: 2 leader_uuid: "3974cce730fa4449ac372e4768750bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } } }
I20260923 22:10:14.732461 16327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:14.732482 16324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:14.733234 16327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:14.733340 16327 catalog_manager.cc:1259] Loaded cluster ID: 7c973244d71e472c9188f234fe8ad6ac
I20260923 22:10:14.733378 16327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:14.733764 16327 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260923 22:10:14.738487 16327 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260923 22:10:14.740967 16327 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:14.741165 16327 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Loaded TSK: 0
I20260923 22:10:14.741369 16327 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:10:14.803937 16321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:14.804138 16321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:14.804162 16321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:14.805356 16321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:14.805394 16321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:14.805440 16321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:10:14.806911 16321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:36263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=42673
--tserver_master_addrs=127.1.244.254:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:14.807144 16321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:14.807406 16321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:14.813004 16346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.813004 16347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:14.813089 16321 server_base.cc:1034] running on GCE node
W20260923 22:10:14.813210 16349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:14.813416 16321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:14.813719 16321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:14.814880 16321 hybrid_clock.cc:648] HybridClock initialized: now 1790201414814863 us; error 35 us; skew 500 ppm
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:14.817266 16321 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:10:14.817745 16321 webserver.cc:460] Webserver started at http://127.1.244.193:42673/ using document root <none> and password file <none>
I20260923 22:10:14.817988 16321 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:14.818065 16321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:14.819343 16321 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:14.820083 16356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.820232 16321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.820333 16321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "a974b8fb65114f9889164c24543f8d76"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "24bf8451f0882c1461d01ae569b211a4"
server_key_iv: "9a6638bde24fba672a1e8158bf32e7af"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.820750 16321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:14.846797 16321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:14.849033 16321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:14.849181 16321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:14.849452 16321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:14.849925 16363 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:10:14.851415 16321 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:10:14.851478 16321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:10:14.851502 16321 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:10:14.852185 16321 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:10:14.852231 16321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:14.852272 16363 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Bootstrap starting.
I20260923 22:10:14.859848 16321 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:36263
I20260923 22:10:14.859889 16470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:36263 every 8 connection(s)
I20260923 22:10:14.860352 16321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:14.863278 16363 log.cc:824] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:14.865271 16363 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:14.865588 16363 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Bootstrap complete.
I20260923 22:10:14.866370 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 16321
I20260923 22:10:14.866664 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:37601
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:43429
--webserver_port=33275
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:14.866706 16363 ts_tablet_manager.cc:1397] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:14.868284 16363 raft_consensus.cc:348] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.868654 16363 raft_consensus.cc:729] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a974b8fb65114f9889164c24543f8d76, State: Initialized, Role: FOLLOWER
I20260923 22:10:14.868808 16363 consensus_queue.cc:260] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:14.869093 16363 ts_tablet_manager.cc:1428] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
I20260923 22:10:14.871531 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.861131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:56519)
0923 22:10:14.861347 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.861396 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.861911 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.862347 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.862351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.862629 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:14.862821 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.862825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.864163 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.864164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.864920 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.864922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.865573 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.865575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.865717 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.866320 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:14.866328 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:14.867882 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:14.869925 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.869931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.869932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.870159 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.870425 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.870426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.870427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.870533 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.870894 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:14.870901 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:14.871036 (+ 135us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.871150 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.871321 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
I20260923 22:10:14.871531 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.861203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:56519 (local address 127.1.244.254:43429)
0923 22:10:14.861396 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.861405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.861522 (+ 117us) server_negotiation.cc:407] Connection header received
0923 22:10:14.862138 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.862143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.862185 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.862321 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:14.862933 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.864026 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.865013 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.865465 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.865590 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.868020 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:14.868024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:14.868025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:14.868039 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:14.869836 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.870256 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.870261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.870262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.870348 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.870613 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.870614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.870614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.870817 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:14.870917 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.871203 (+ 286us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.871325 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260923 22:10:14.872682 16471 heartbeater.cc:347] Connected to a master server at 127.1.244.254:43429
I20260923 22:10:14.872757 16471 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:14.872939 16471 heartbeater.cc:511] Master 127.1.244.254:43429 requested a full tablet report, sending...
I20260923 22:10:14.873613 16283 ts_manager.cc:194] Registered new tserver with Master: a974b8fb65114f9889164c24543f8d76 (127.1.244.193:36263)
I20260923 22:10:14.874714 16283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:56519
I20260923 22:10:14.875537 16471 heartbeater.cc:503] Master 127.1.244.254:43429 was elected leader, sending a full tablet report...
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:10:14.882189 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.874140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36263 (local address 127.0.0.1:58380)
0923 22:10:14.874320 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:14.874366 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:14.874434 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:14.874679 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:14.874685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:14.875054 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:14.875116 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.875119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.876551 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.876552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.877086 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:14.877088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.877706 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.877707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:14.877757 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.878187 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:14.878191 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:14.879389 (+ 1198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:14.881217 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.881223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.881224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.881402 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.881622 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:14.881623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:14.881623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:14.881672 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:14.881948 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:14.881952 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:14.881989 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 22:10:14.882048 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:10:14.882129 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260923 22:10:14.882280 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:14.874372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.1.244.193:36263)
0923 22:10:14.874524 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:14.874528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:14.874538 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:14.874567 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:14.874569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:14.874580 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:14.874656 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:14.875237 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.876473 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.877175 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:14.877628 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:14.877701 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:14.879468 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:14.879475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:14.879475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:14.879482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:14.881145 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.881477 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.881480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.881480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.881559 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:14.881728 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:14.881729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:14.881729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:14.881888 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:14.881964 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:14.882103 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 22:10:14.882190 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260923 22:10:14.882571 16405 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f8f52fdb764b34ad31abd367e728f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:14 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:58380
I20260923 22:10:14.883062 16480 tablet_replica.cc:330] stopping tablet replica
I20260923 22:10:14.883167 16480 raft_consensus.cc:2229] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:14.883231 16480 raft_consensus.cc:2258] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:14.883499 16480 ts_tablet_manager.cc:1905] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:14.885380 16480 ts_tablet_manager.cc:1918] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:14.885438 16480 log.cc:1196] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/d4f8f52fdb764b34ad31abd367e728f0
I20260923 22:10:14.885669 16480 ts_tablet_manager.cc:1939] T d4f8f52fdb764b34ad31abd367e728f0 P a974b8fb65114f9889164c24543f8d76: Deleting consensus metadata
I20260923 22:10:14.886130 16270 catalog_manager.cc:4841] TS a974b8fb65114f9889164c24543f8d76 (127.1.244.193:36263): tablet d4f8f52fdb764b34ad31abd367e728f0 (table test-table [id=8ea619aec55b4c248d572750a72d310f]) successfully deleted
W20260923 22:10:14.952809 16476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:14.953034 16476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:14.953071 16476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:14.954187 16476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:14.954253 16476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:14.954298 16476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:10:14.955837 16476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:37601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=33275
--tserver_master_addrs=127.1.244.254:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:14.956115 16476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:14.956333 16476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:14.961258 16486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:14.961361 16476 server_base.cc:1034] running on GCE node
W20260923 22:10:14.961272 16488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:14.961263 16485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:14.961736 16476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:14.962037 16476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:14.963213 16476 hybrid_clock.cc:648] HybridClock initialized: now 1790201414963210 us; error 35 us; skew 500 ppm
Sep 23 22:10:14 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:14.965801 16476 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:10:14.966241 16476 webserver.cc:460] Webserver started at http://127.1.244.194:33275/ using document root <none> and password file <none>
I20260923 22:10:14.966485 16476 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:14.966572 16476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:14.968073 16476 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:10:14.968978 16495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:14.969156 16476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:14.969266 16476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "711362bc8ff54e03922d2fac51a3ffe0"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "59243b5e4883f635826aabcf0bcf4f50"
server_key_iv: "417d686676e63c44608dc24dd11de57a"
server_key_version: "encryptionkey@0"
I20260923 22:10:14.969651 16476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:14.994668 16476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:14.996734 16476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:14.996901 16476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:14.997133 16476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:14.997576 16502 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:10:14.998939 16476 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:10:14.999010 16476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:10:14.999056 16476 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:10:14.999789 16476 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:10:14.999848 16476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:14.999898 16502 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Bootstrap starting.
I20260923 22:10:15.001739 16502 log.cc:824] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:15.005380 16502 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:15.005743 16502 tablet_bootstrap.cc:492] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Bootstrap complete.
I20260923 22:10:15.005924 16502 ts_tablet_manager.cc:1397] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260923 22:10:15.007099 16476 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:37601
I20260923 22:10:15.007177 16610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:37601 every 8 connection(s)
I20260923 22:10:15.007503 16476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:15.008307 16502 raft_consensus.cc:348] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:15.008675 16502 raft_consensus.cc:729] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711362bc8ff54e03922d2fac51a3ffe0, State: Initialized, Role: FOLLOWER
I20260923 22:10:15.008878 16502 consensus_queue.cc:260] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } }
I20260923 22:10:15.009176 16502 ts_tablet_manager.cc:1428] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:10:15.012132 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 16476
I20260923 22:10:15.012473 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:39763
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:43429
--webserver_port=38955
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:10:15 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:15.017203 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.008251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:58461 (local address 127.1.244.254:43429)
0923 22:10:15.008323 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.008326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.008537 (+ 211us) server_negotiation.cc:407] Connection header received
0923 22:10:15.008737 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.008739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.008750 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.008823 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:15.009611 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.010108 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.011148 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.011342 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.011393 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.014276 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:15.014277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:15.014278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:15.014285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:15.015790 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.016231 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.016232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.016233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.016292 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.016552 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.016553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.016553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.016672 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:15.016744 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.017012 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.017102 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:10:15.017388 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.008209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:58461)
0923 22:10:15.008388 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.008433 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:15.008624 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:15.008880 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:15.008885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:15.009301 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:15.009526 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.009531 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.010219 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.010220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.011070 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.011072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.011476 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.011478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.011659 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.012627 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:15.012637 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:15.014164 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:15.015905 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:15.015911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:15.015913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:15.016141 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:15.016359 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:15.016360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:15.016360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:15.016479 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:15.016746 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:15.016753 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:15.016886 (+ 133us) client_negotiation.cc:769] Sending connection context
0923 22:10:15.016991 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:10:15.017145 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260923 22:10:15.017884 16611 heartbeater.cc:347] Connected to a master server at 127.1.244.254:43429
I20260923 22:10:15.018010 16611 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:15.018200 16611 heartbeater.cc:511] Master 127.1.244.254:43429 requested a full tablet report, sending...
I20260923 22:10:15.018736 16283 ts_manager.cc:194] Registered new tserver with Master: 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601)
I20260923 22:10:15.019287 16283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:58461
I20260923 22:10:15.020212 16611 heartbeater.cc:503] Master 127.1.244.254:43429 was elected leader, sending a full tablet report...
Sep 23 22:10:15 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201414, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260923 22:10:15.026602 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.019051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37601 (local address 127.0.0.1:37542)
0923 22:10:15.019105 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.019116 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:15.019158 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:15.019579 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:15.019580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:15.019736 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:15.019773 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.019774 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.020805 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.020807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.021353 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.021355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.021847 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.021849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.021915 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.022338 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:15.022342 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:15.023579 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:15.025530 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:15.025531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:15.025532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:15.025692 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:15.025918 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:15.025919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:15.025919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:15.025969 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:15.026271 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:15.026272 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:15.026316 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 22:10:15.026402 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:10:15.026506 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:10:15.026607 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.019124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37542 (local address 127.1.244.194:37601)
0923 22:10:15.019276 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.019281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.019290 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:10:15.019325 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.019328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.019342 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.019529 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:15.019864 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.020745 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.021449 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.021769 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.021850 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.023667 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:15.023674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:15.023674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:15.023681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:15.025448 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.025768 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.025771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.025772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.025853 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.026040 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.026041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.026041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.026190 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:15.026278 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.026435 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.026523 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260923 22:10:15.027139 16545 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f8f52fdb764b34ad31abd367e728f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:14 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:37542
I20260923 22:10:15.027681 16617 tablet_replica.cc:330] stopping tablet replica
I20260923 22:10:15.027784 16617 raft_consensus.cc:2229] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:15.027848 16617 raft_consensus.cc:2258] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:15.028137 16617 ts_tablet_manager.cc:1905] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:15.030490 16617 ts_tablet_manager.cc:1918] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:15.030565 16617 log.cc:1196] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/d4f8f52fdb764b34ad31abd367e728f0
I20260923 22:10:15.030789 16617 ts_tablet_manager.cc:1939] T d4f8f52fdb764b34ad31abd367e728f0 P 711362bc8ff54e03922d2fac51a3ffe0: Deleting consensus metadata
I20260923 22:10:15.031294 16270 catalog_manager.cc:4841] TS 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601): tablet d4f8f52fdb764b34ad31abd367e728f0 (table test-table [id=8ea619aec55b4c248d572750a72d310f]) successfully deleted
W20260923 22:10:15.097826 16615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:15.098028 16615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:15.098052 16615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:15.099227 16615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:15.099269 16615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:15.099340 16615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:10:15.100848 16615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:39763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=38955
--tserver_master_addrs=127.1.244.254:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:15.101081 16615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:15.101260 16615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:15.106154 16622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:15.106171 16623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:15.106189 16615 server_base.cc:1034] running on GCE node
W20260923 22:10:15.106137 16625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:15.106511 16615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:15.106755 16615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:15.107888 16615 hybrid_clock.cc:648] HybridClock initialized: now 1790201415107878 us; error 26 us; skew 500 ppm
Sep 23 22:10:15 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:15.110273 16615 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:10:15.110736 16615 webserver.cc:460] Webserver started at http://127.1.244.195:38955/ using document root <none> and password file <none>
I20260923 22:10:15.110965 16615 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:15.111047 16615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:15.112418 16615 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:15.112973 16632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:15.113160 16615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:15.113248 16615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "61f8947b4bab481494a1bcefda29cec3"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "635cd07c6dc1d859202610d857cb6e75"
server_key_iv: "27e0043aac6e22d39861aab373f6fee7"
server_key_version: "encryptionkey@0"
I20260923 22:10:15.113556 16615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:15.123961 16615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:15.125644 16615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:15.125793 16615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:15.126003 16615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:15.126291 16615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:15.126322 16615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:15.126363 16615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:15.126389 16615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:15.131913 16615 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:39763
I20260923 22:10:15.131969 16745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:39763 every 8 connection(s)
I20260923 22:10:15.132295 16615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:15.137532 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 16615
Sep 23 22:10:15 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:15.141916 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.133048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:52463 (local address 127.1.244.254:43429)
0923 22:10:15.133128 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.133130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.133328 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:10:15.133891 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.133893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.133905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.133973 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:15.134894 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.135384 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.136374 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.136594 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.136652 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.138797 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:15.138798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:15.138798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:15.138806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:15.140465 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.140878 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.140879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.140880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.140948 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.141195 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.141196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.141196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.141336 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:15.141393 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.141730 (+ 337us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.141828 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:10:15.142061 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.133019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:52463)
0923 22:10:15.133188 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.133220 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:15.133772 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:15.134058 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:15.134061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:15.134609 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:15.134793 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.134797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.135461 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.135462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.136289 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.136291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.136668 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.136669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.136764 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.137190 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:15.137198 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:15.138696 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:15.140573 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:15.140579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:15.140581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:15.140798 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:15.141018 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:15.141019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:15.141019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:15.141124 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:15.141401 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:15.141406 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:15.141567 (+ 161us) client_negotiation.cc:769] Sending connection context
0923 22:10:15.141675 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 22:10:15.141794 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":57,"threads_started":1}
I20260923 22:10:15.142545 16746 heartbeater.cc:347] Connected to a master server at 127.1.244.254:43429
I20260923 22:10:15.142669 16746 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:15.142874 16746 heartbeater.cc:511] Master 127.1.244.254:43429 requested a full tablet report, sending...
I20260923 22:10:15.143355 16283 ts_manager.cc:194] Registered new tserver with Master: 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763)
I20260923 22:10:15.143777 16283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:52463
I20260923 22:10:15.145670 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.138486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34078 (local address 127.1.244.254:43429)
0923 22:10:15.138657 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.138659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.138668 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:10:15.138692 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.138693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.138704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.138773 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:15.139513 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.140148 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.141252 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.141493 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.141561 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.143028 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:15.143029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:15.143030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:15.143037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:15.144746 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.145064 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.145064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.145065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.145117 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.145266 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.145267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.145267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.145357 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:15.145474 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.145507 (+ 33us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.145571 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260923 22:10:15.146505 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:15.149089 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.146851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.1.244.254:43429)
0923 22:10:15.146886 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.146888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.146984 (+ 96us) server_negotiation.cc:407] Connection header received
0923 22:10:15.147041 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.147042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.147068 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.147128 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:15.147198 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.147762 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.148479 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.148680 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.148735 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.148853 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:15.148941 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:15.149003 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.149022 (+ 19us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.149041 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:10:15.149360 16283 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:15.150944 16283 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:15.153129 16545 tablet_service.cc:1467] Processing CreateTablet for tablet bffd6be7273945598129683960acaf9c (DEFAULT_TABLE table=test-table [id=46158e5d6e0e45ba918f6a868b31488c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:15.153126 16405 tablet_service.cc:1467] Processing CreateTablet for tablet bffd6be7273945598129683960acaf9c (DEFAULT_TABLE table=test-table [id=46158e5d6e0e45ba918f6a868b31488c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:15.153327 16545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bffd6be7273945598129683960acaf9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:15.153323 16405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bffd6be7273945598129683960acaf9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:15.154727 16363 tablet_bootstrap.cc:492] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Bootstrap starting.
I20260923 22:10:15.154765 16502 tablet_bootstrap.cc:492] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: Bootstrap starting.
I20260923 22:10:15.155093 16363 tablet_bootstrap.cc:654] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:15.155189 16502 tablet_bootstrap.cc:654] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:15.155715 16363 tablet_bootstrap.cc:492] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: No bootstrap required, opened a new log
I20260923 22:10:15.155776 16363 ts_tablet_manager.cc:1397] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:15.155895 16502 tablet_bootstrap.cc:492] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: No bootstrap required, opened a new log
I20260923 22:10:15.155946 16502 ts_tablet_manager.cc:1397] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:15.156083 16502 raft_consensus.cc:348] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.156149 16502 raft_consensus.cc:374] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:15.156195 16502 raft_consensus.cc:729] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711362bc8ff54e03922d2fac51a3ffe0, State: Initialized, Role: FOLLOWER
I20260923 22:10:15.156191 16363 raft_consensus.cc:348] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.156237 16502 consensus_queue.cc:260] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.156275 16363 raft_consensus.cc:374] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:15.156327 16363 raft_consensus.cc:729] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a974b8fb65114f9889164c24543f8d76, State: Initialized, Role: FOLLOWER
I20260923 22:10:15.156335 16502 ts_tablet_manager.cc:1428] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:15.156376 16363 consensus_queue.cc:260] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.156495 16363 ts_tablet_manager.cc:1428] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:15.158849 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.152781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39763 (local address 127.0.0.1:54870)
0923 22:10:15.152831 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.152843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:15.152889 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:15.155023 (+ 2134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:15.155024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:15.155026 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:15.155070 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.155072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.156800 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.156802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.157816 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.157818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.158582 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.158583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.158651 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.158672 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 22:10:15.158738 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:10:15.158779 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:10:15.158878 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.152901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54870 (local address 127.1.244.195:39763)
0923 22:10:15.154841 (+ 1940us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.154846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.154857 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:10:15.154900 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.154902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.154921 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.155021 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:15.155143 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.156258 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.157912 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.158483 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.158585 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.158647 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.158753 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.158795 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1905,"thread_start_us":84,"threads_started":1}
I20260923 22:10:15.159633 16680 tablet_service.cc:1467] Processing CreateTablet for tablet bffd6be7273945598129683960acaf9c (DEFAULT_TABLE table=test-table [id=46158e5d6e0e45ba918f6a868b31488c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:15.159929 16680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bffd6be7273945598129683960acaf9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:15.162761 16755 tablet_bootstrap.cc:492] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: Bootstrap starting.
I20260923 22:10:15.163308 16755 tablet_bootstrap.cc:654] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:15.163810 16755 log.cc:824] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:15.164491 16755 tablet_bootstrap.cc:492] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: No bootstrap required, opened a new log
I20260923 22:10:15.164577 16755 ts_tablet_manager.cc:1397] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:10:15.166221 16755 raft_consensus.cc:348] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.166388 16755 raft_consensus.cc:374] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:15.166435 16755 raft_consensus.cc:729] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f8947b4bab481494a1bcefda29cec3, State: Initialized, Role: FOLLOWER
I20260923 22:10:15.166597 16755 consensus_queue.cc:260] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.166878 16755 ts_tablet_manager.cc:1428] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:10:15.166890 16746 heartbeater.cc:503] Master 127.1.244.254:43429 was elected leader, sending a full tablet report...
W20260923 22:10:15.258085 16612 tablet.cc:2369] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:10:15.361133 16472 tablet.cc:2369] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:10:15.382881 16747 tablet.cc:2369] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:15.519797 16757 raft_consensus.cc:482] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:15.519924 16757 raft_consensus.cc:504] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.520367 16757 leader_election.cc:288] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601), a974b8fb65114f9889164c24543f8d76 (127.1.244.193:36263)
I20260923 22:10:15.524480 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.520544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37601 (local address 127.1.244.195:45289)
0923 22:10:15.520610 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.520623 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:15.520704 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:15.520907 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:15.520907 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:15.520909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:15.520990 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.520992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.522003 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.522003 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.523329 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.523332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.524200 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.524201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.524289 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.524291 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:10:15.524342 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:10:15.524384 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":57}
I20260923 22:10:15.524504 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.520754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:48697 (local address 127.1.244.193:36263)
0923 22:10:15.520888 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.520890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.520902 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:10:15.520945 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.520946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.520978 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.521062 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:15.521195 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.521896 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.523444 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.524109 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.524194 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.524293 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.524355 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.524397 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260923 22:10:15.524495 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.520664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:45289 (local address 127.1.244.194:37601)
0923 22:10:15.520738 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.520740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.520750 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:15.520781 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.520782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.520809 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.520883 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:15.521068 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.521897 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.523437 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.524116 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.524255 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.524319 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.524348 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.524394 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:10:15.524487 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.520661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36263 (local address 127.1.244.195:48697)
0923 22:10:15.520827 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.520835 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:15.520877 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:15.521062 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:15.521063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:15.521064 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:15.521126 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.521127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.521991 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.521992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.523329 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:15.523332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.524201 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.524201 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:15.524287 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.524291 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:15.524351 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:10:15.524384 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":42,"thread_start_us":95,"threads_started":1}
I20260923 22:10:15.524920 16565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bffd6be7273945598129683960acaf9c" candidate_uuid: "61f8947b4bab481494a1bcefda29cec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "711362bc8ff54e03922d2fac51a3ffe0" is_pre_election: true
I20260923 22:10:15.524971 16412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bffd6be7273945598129683960acaf9c" candidate_uuid: "61f8947b4bab481494a1bcefda29cec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a974b8fb65114f9889164c24543f8d76" is_pre_election: true
I20260923 22:10:15.525111 16565 raft_consensus.cc:2454] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61f8947b4bab481494a1bcefda29cec3 in term 0.
I20260923 22:10:15.525156 16412 raft_consensus.cc:2454] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61f8947b4bab481494a1bcefda29cec3 in term 0.
I20260923 22:10:15.525380 16636 leader_election.cc:302] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61f8947b4bab481494a1bcefda29cec3, 711362bc8ff54e03922d2fac51a3ffe0; no voters:
I20260923 22:10:15.525548 16757 raft_consensus.cc:2781] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:15.525629 16757 raft_consensus.cc:482] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:15.525648 16757 raft_consensus.cc:3037] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:15.526362 16757 raft_consensus.cc:504] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.526499 16757 leader_election.cc:288] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [CANDIDATE]: Term 1 election: Requested vote from peers 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601), a974b8fb65114f9889164c24543f8d76 (127.1.244.193:36263)
I20260923 22:10:15.526705 16565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bffd6be7273945598129683960acaf9c" candidate_uuid: "61f8947b4bab481494a1bcefda29cec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "711362bc8ff54e03922d2fac51a3ffe0"
I20260923 22:10:15.526742 16412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bffd6be7273945598129683960acaf9c" candidate_uuid: "61f8947b4bab481494a1bcefda29cec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a974b8fb65114f9889164c24543f8d76"
I20260923 22:10:15.526785 16565 raft_consensus.cc:3037] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:15.526849 16412 raft_consensus.cc:3037] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:15.527494 16565 raft_consensus.cc:2454] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61f8947b4bab481494a1bcefda29cec3 in term 1.
I20260923 22:10:15.527535 16412 raft_consensus.cc:2454] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61f8947b4bab481494a1bcefda29cec3 in term 1.
I20260923 22:10:15.527686 16636 leader_election.cc:302] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61f8947b4bab481494a1bcefda29cec3, 711362bc8ff54e03922d2fac51a3ffe0; no voters:
I20260923 22:10:15.527807 16757 raft_consensus.cc:2781] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:15.527961 16757 raft_consensus.cc:686] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 1 LEADER]: Becoming Leader. State: Replica: 61f8947b4bab481494a1bcefda29cec3, State: Running, Role: LEADER
I20260923 22:10:15.528072 16757 consensus_queue.cc:237] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.528888 16283 catalog_manager.cc:5495] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "61f8947b4bab481494a1bcefda29cec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260923 22:10:15.597450 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.594921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54878 (local address 127.1.244.195:39763)
0923 22:10:15.594991 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.594994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.595003 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:10:15.595089 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.595090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.595109 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.595190 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:15.595324 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.595877 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.596657 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.596849 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.596925 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.597084 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:15.597189 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:15.597291 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.597313 (+ 22us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.597334 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:10:15.599323 16412 raft_consensus.cc:1261] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Refusing update from remote peer 61f8947b4bab481494a1bcefda29cec3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:15.599323 16565 raft_consensus.cc:1261] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Refusing update from remote peer 61f8947b4bab481494a1bcefda29cec3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:15.599732 16760 consensus_queue.cc:1035] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:15.599931 16757 consensus_queue.cc:1035] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:15.601879 16763 mvcc.cc:205] Tried to move back new op lower bound from 7332664998291787776 to 7332664998003585024. Current Snapshot: MvccSnapshot[applied={T|T < 7332664998291787776}]
I20260923 22:10:15.604406 16283 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:15.604509 16283 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:15.605259 16283 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Sending DeleteTablet for 3 replicas of tablet bffd6be7273945598129683960acaf9c
I20260923 22:10:15.605587 16405 tablet_service.cc:1514] Processing DeleteTablet for tablet bffd6be7273945598129683960acaf9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:15 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:58380
I20260923 22:10:15.605593 16545 tablet_service.cc:1514] Processing DeleteTablet for tablet bffd6be7273945598129683960acaf9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:15 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:37542
I20260923 22:10:15.605633 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 16321
I20260923 22:10:15.606379 16771 tablet_replica.cc:330] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: stopping tablet replica
I20260923 22:10:15.606448 16771 raft_consensus.cc:2229] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:15.606501 16771 raft_consensus.cc:2258] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:15.606820 16771 ts_tablet_manager.cc:1905] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:15.608383 16680 tablet_service.cc:1514] Processing DeleteTablet for tablet bffd6be7273945598129683960acaf9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:15 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:54870
I20260923 22:10:15.608748 16771 ts_tablet_manager.cc:1918] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:15.608803 16771 log.cc:1196] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/bffd6be7273945598129683960acaf9c
I20260923 22:10:15.608820 16772 tablet_replica.cc:330] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: stopping tablet replica
I20260923 22:10:15.608913 16772 raft_consensus.cc:2229] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:10:15.609009 16772 raft_consensus.cc:2258] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:15.609030 16771 ts_tablet_manager.cc:1939] T bffd6be7273945598129683960acaf9c P 711362bc8ff54e03922d2fac51a3ffe0: Deleting consensus metadata
I20260923 22:10:15.609338 16270 catalog_manager.cc:4841] TS 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601): tablet bffd6be7273945598129683960acaf9c (table test-table [id=46158e5d6e0e45ba918f6a868b31488c]) successfully deleted
I20260923 22:10:15.609542 16772 ts_tablet_manager.cc:1905] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:15.610648 16772 ts_tablet_manager.cc:1918] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:15.610694 16772 log.cc:1196] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/bffd6be7273945598129683960acaf9c
I20260923 22:10:15.610891 16772 ts_tablet_manager.cc:1939] T bffd6be7273945598129683960acaf9c P 61f8947b4bab481494a1bcefda29cec3: Deleting consensus metadata
I20260923 22:10:15.611205 16270 catalog_manager.cc:4841] TS 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763): tablet bffd6be7273945598129683960acaf9c (table test-table [id=46158e5d6e0e45ba918f6a868b31488c]) successfully deleted
W20260923 22:10:15.612502 16270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:36263 (error 108)
I20260923 22:10:15.612882 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 16476
I20260923 22:10:15.612957 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.612816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36263 (local address 127.0.0.1:58396)
0923 22:10:15.612865 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.612918 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:10:15.614486 16270 catalog_manager.cc:4584] TS a974b8fb65114f9889164c24543f8d76 (127.1.244.193:36263): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bffd6be7273945598129683960acaf9c: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36263: connect: Connection refused (error 111)
I20260923 22:10:15.618532 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 16615
I20260923 22:10:15.624224 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 16250
I20260923 22:10:15.629451 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:43429
--webserver_interface=127.1.244.254
--webserver_port=32791
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:43429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:15.709362 16774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:15.709574 16774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:15.709596 16774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:15.710657 16774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:15.710690 16774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:15.710704 16774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:15.710716 16774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:15.710729 16774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:15.712145 16774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:43429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:43429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=32791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:15.712323 16774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:15.712500 16774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:15.717146 16779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:15.717191 16782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:15.717303 16774 server_base.cc:1034] running on GCE node
W20260923 22:10:15.717204 16780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:15.717778 16774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:15.718079 16774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:15.719244 16774 hybrid_clock.cc:648] HybridClock initialized: now 1790201415719220 us; error 38 us; skew 500 ppm
Sep 23 22:10:15 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:15.722038 16774 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:10:15.722576 16774 webserver.cc:460] Webserver started at http://127.1.244.254:32791/ using document root <none> and password file <none>
I20260923 22:10:15.722815 16774 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:15.722867 16774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:15.724274 16774 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:15.724941 16789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:15.725123 16774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:15.725229 16774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "3974cce730fa4449ac372e4768750bb9"
format_stamp: "Formatted at 2026-09-23 22:10:13 on dist-test-slave-b5jz"
server_key: "bcb312dc48debf28415f3feeb4ec9b5e"
server_key_iv: "0306c5b17aa2a2095b678cbb5ffce1a8"
server_key_version: "encryptionkey@0"
I20260923 22:10:15.725586 16774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:15.734699 16774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:15.736883 16774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:15.737044 16774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:15.740885 16774 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:43429
I20260923 22:10:15.740931 16841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:43429 every 8 connection(s)
I20260923 22:10:15.741300 16774 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:15.742762 16842 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:10:15.743402 16842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Bootstrap starting.
I20260923 22:10:15.744088 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 16774
I20260923 22:10:15.744383 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:36263
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:43429
--webserver_port=42673
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:15.752175 16842 log.cc:824] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:15.754302 16842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:15.754603 16842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9: Bootstrap complete.
I20260923 22:10:15.756807 16842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:15.757048 16842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3974cce730fa4449ac372e4768750bb9, State: Initialized, Role: FOLLOWER
I20260923 22:10:15.757232 16842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:15.757310 16842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:15.757362 16842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:15.757416 16842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:10:15.758477 16842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:15.758598 16842 leader_election.cc:302] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3974cce730fa4449ac372e4768750bb9; no voters:
I20260923 22:10:15.758792 16842 leader_election.cc:288] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 22:10:15.758826 16846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:10:15.758927 16846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [term 3 LEADER]: Becoming Leader. State: Replica: 3974cce730fa4449ac372e4768750bb9, State: Running, Role: LEADER
I20260923 22:10:15.759096 16846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } }
I20260923 22:10:15.759397 16848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3974cce730fa4449ac372e4768750bb9. Latest consensus state: current_term: 3 leader_uuid: "3974cce730fa4449ac372e4768750bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } } }
I20260923 22:10:15.759686 16848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:15.759719 16847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3974cce730fa4449ac372e4768750bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3974cce730fa4449ac372e4768750bb9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 43429 } } }
I20260923 22:10:15.759828 16847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:15.759085 16842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3974cce730fa4449ac372e4768750bb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:15.759961 16851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:15.760775 16851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:15.760955 16851 catalog_manager.cc:1259] Loaded cluster ID: 7c973244d71e472c9188f234fe8ad6ac
I20260923 22:10:15.761081 16851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260923 22:10:15.761529 16851 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f79a00209ad google::LogMessage::Fail() at ??:0
@ 0x7f79a002388a google::LogMessage::SendToLog() at ??:0
@ 0x7f79a00204de google::LogMessage::Flush() at ??:0
@ 0x7f79a0021a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f79a7f450f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f79a7f4e8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f79a1eee0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f79a1ee3954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f79a2d476db start_thread at ??:0
@ 0x7f799eb1e61f clone at ??:0
W20260923 22:10:15.841997 16844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:15.842200 16844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:15.842224 16844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:15.843372 16844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:15.843415 16844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:15.843441 16844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:10:15.844883 16844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:36263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=42673
--tserver_master_addrs=127.1.244.254:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:15.845118 16844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:15.845360 16844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:15.851133 16881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:15.851162 16879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:15.851306 16878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:15.851572 16844 server_base.cc:1034] running on GCE node
I20260923 22:10:15.851804 16844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:15.852066 16844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:15.853238 16844 hybrid_clock.cc:648] HybridClock initialized: now 1790201415853225 us; error 32 us; skew 500 ppm
Sep 23 22:10:15 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:15.855731 16844 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:10:15.856150 16844 webserver.cc:460] Webserver started at http://127.1.244.193:42673/ using document root <none> and password file <none>
I20260923 22:10:15.856376 16844 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:15.856448 16844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:15.857822 16844 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:15.858541 16888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:15.858722 16844 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:15.858819 16844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "a974b8fb65114f9889164c24543f8d76"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "24bf8451f0882c1461d01ae569b211a4"
server_key_iv: "9a6638bde24fba672a1e8158bf32e7af"
server_key_version: "encryptionkey@0"
I20260923 22:10:15.859161 16844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:15.885033 16844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:15.887517 16844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:15.887735 16844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:15.887989 16844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:15.888458 16895 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:10:15.890491 16844 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:10:15.890551 16844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:10:15.890625 16844 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:10:15.891278 16844 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:10:15.891335 16844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:15.891412 16895 tablet_bootstrap.cc:492] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Bootstrap starting.
I20260923 22:10:15.899092 16844 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:36263
I20260923 22:10:15.899155 17002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:36263 every 8 connection(s)
I20260923 22:10:15.899636 16844 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:15.900007 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 16844
I20260923 22:10:15.900307 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:37601
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:43429
--webserver_port=33275
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:15.902463 16895 log.cc:824] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:15.905171 16895 tablet_bootstrap.cc:492] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:15.905501 16895 tablet_bootstrap.cc:492] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Bootstrap complete.
I20260923 22:10:15.906714 16895 ts_tablet_manager.cc:1397] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260923 22:10:15.908131 16895 raft_consensus.cc:348] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.908455 16895 raft_consensus.cc:729] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a974b8fb65114f9889164c24543f8d76, State: Initialized, Role: FOLLOWER
I20260923 22:10:15.908619 16895 consensus_queue.cc:260] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:15.908857 16895 ts_tablet_manager.cc:1428] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260923 22:10:15.942983 17005 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:10:15.900623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:34991)
0923 22:10:15.900885 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.900927 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:15.901632 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:15.942762 (+ 41130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":97,"threads_started":1}
W20260923 22:10:15.943202 17003 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:43429 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:43429: Client connection negotiation failed: client connection to 127.1.244.254:43429: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:10:15.943605 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.943432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:45087)
0923 22:10:15.943520 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.943581 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260923 22:10:15.943984 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.943890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:53223)
0923 22:10:15.943938 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:15.943964 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 22:10:15.944116 17003 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:10:15.984148 17006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:15.984349 17006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:15.984373 17006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:15.985476 17006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:15.985513 17006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:15.985539 17006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:10:15.987012 17006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:37601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=33275
--tserver_master_addrs=127.1.244.254:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:15.987207 17006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:15.987380 17006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:15.992391 17014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:15.992535 17006 server_base.cc:1034] running on GCE node
W20260923 22:10:15.992399 17016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:15.992399 17013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:15.992867 17006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:15.993124 17006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:15.994289 17006 hybrid_clock.cc:648] HybridClock initialized: now 1790201415994285 us; error 38 us; skew 500 ppm
Sep 23 22:10:15 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:15.996858 17006 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:10:15.997308 17006 webserver.cc:460] Webserver started at http://127.1.244.194:33275/ using document root <none> and password file <none>
I20260923 22:10:15.997486 17006 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:15.997527 17006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:15.998776 17006 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:15.999325 17023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:15.999583 17006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:15.999676 17006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "711362bc8ff54e03922d2fac51a3ffe0"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "59243b5e4883f635826aabcf0bcf4f50"
server_key_iv: "417d686676e63c44608dc24dd11de57a"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.000016 17006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:16.014535 17006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:16.016342 17006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:16.016448 17006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:16.016633 17006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:16.016860 17006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:16.016888 17006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.016909 17006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:16.016921 17006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.022496 17006 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:37601
I20260923 22:10:16.022634 17136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:37601 every 8 connection(s)
I20260923 22:10:16.022879 17006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:16.023979 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.023599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:35895)
0923 22:10:16.023773 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.023840 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
W20260923 22:10:16.024386 17137 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:43429 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:43429: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
I20260923 22:10:16.024688 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.024590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:40619)
0923 22:10:16.024638 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.024668 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:10:16.025004 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.024913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:48931)
0923 22:10:16.024962 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.024984 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:10:16.025137 17137 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:10:16.026456 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 17006
I20260923 22:10:16.026723 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:39763
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:43429
--webserver_port=38955
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:16.107685 17140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:16.107879 17140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:16.107903 17140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:16.109047 17140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:16.109092 17140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:16.109126 17140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:10:16.110544 17140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:39763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=38955
--tserver_master_addrs=127.1.244.254:43429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:16.110785 17140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:16.111032 17140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:16.115895 17148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:16.115902 17145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:16.116020 17146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:16.116080 17140 server_base.cc:1034] running on GCE node
I20260923 22:10:16.116297 17140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:16.116532 17140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:16.117697 17140 hybrid_clock.cc:648] HybridClock initialized: now 1790201416117686 us; error 31 us; skew 500 ppm
Sep 23 22:10:16 dist-test-slave-b5jz krb5kdc[15703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:16.120230 17140 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:10:16.120674 17140 webserver.cc:460] Webserver started at http://127.1.244.195:38955/ using document root <none> and password file <none>
I20260923 22:10:16.120949 17140 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:16.121021 17140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:16.122314 17140 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:16.122891 17155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.123152 17140 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:16.123256 17140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "61f8947b4bab481494a1bcefda29cec3"
format_stamp: "Formatted at 2026-09-23 22:10:14 on dist-test-slave-b5jz"
server_key: "635cd07c6dc1d859202610d857cb6e75"
server_key_iv: "27e0043aac6e22d39861aab373f6fee7"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.123625 17140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:16.135417 17140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:16.137669 17140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:16.137812 17140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:16.138043 17140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:16.138341 17140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:16.138396 17140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.138435 17140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:16.138463 17140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.144270 17140 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:39763
I20260923 22:10:16.144343 17268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:39763 every 8 connection(s)
I20260923 22:10:16.144685 17140 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:16.145685 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.145321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:53117)
0923 22:10:16.145488 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.145552 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
W20260923 22:10:16.146090 17269 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:43429 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:43429: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
I20260923 22:10:16.146335 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.146254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:42045)
0923 22:10:16.146295 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.146320 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:10:16.146588 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.146519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:57447)
0923 22:10:16.146556 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.146576 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 22:10:16.146701 17269 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:10:16.151392 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 17140
I20260923 22:10:16.944738 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.944517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:34905)
0923 22:10:16.944639 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.944676 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260923 22:10:17.025672 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.025442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:50773)
0923 22:10:17.025556 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.025589 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260923 22:10:17.147228 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.147016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:59113)
0923 22:10:17.147145 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.147180 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":58,"threads_started":1}
I20260923 22:10:17.246006 17276 raft_consensus.cc:482] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:17.246131 17276 raft_consensus.cc:504] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:17.246405 17276 leader_election.cc:288] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763), 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601)
Sep 23 22:10:17 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
Sep 23 22:10:17 dist-test-slave-b5jz krb5kdc[15703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260923 22:10:17.257376 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.246802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:34079 (local address 127.1.244.194:37601)
0923 22:10:17.246977 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.246987 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.247023 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:10:17.247448 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.247450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.247521 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.247679 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:17.248334 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.249191 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.250164 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.250674 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.250785 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.253592 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:17.253595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:17.253596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:17.253605 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:17.255531 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.255953 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.255956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.255956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.256045 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.256375 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.256377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.256377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.256695 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:17.256785 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.257127 (+ 342us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.257261 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":57,"threads_started":1}
I20260923 22:10:17.257376 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.246678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:49145 (local address 127.1.244.195:39763)
0923 22:10:17.246844 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.246860 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.246893 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:10:17.247347 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.247349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.247392 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.247509 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:17.248327 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.249187 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.250186 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.250667 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.250783 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.253158 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:17.253163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:17.253163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:17.253175 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:17.255326 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.255885 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.255888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.255888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.256027 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.256393 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.256394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.256394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.256695 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:17.256785 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.257127 (+ 342us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.257261 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":54,"threads_started":1}
I20260923 22:10:17.257372 17278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.246728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37601 (local address 127.1.244.193:34079)
0923 22:10:17.246840 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.246850 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.246897 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.247678 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.247679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.248015 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:17.248207 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.248213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.249296 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.249297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.250059 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.250061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.250856 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.250857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.250898 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.251476 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:17.251485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:17.253477 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:17.255646 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.255647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.255647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.255865 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.256114 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.256115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.256115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.256276 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.256777 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:17.256784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:17.256938 (+ 154us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.257046 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.257251 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":34,"thread_start_us":63,"threads_started":1}
I20260923 22:10:17.257422 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.246557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39763 (local address 127.1.244.193:49145)
0923 22:10:17.246620 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.246634 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.246710 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.247613 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.247616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.248016 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:17.248206 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.248212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.249316 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.249322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.250060 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.250063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.250758 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.250759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.250878 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.251528 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:17.251532 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:17.253042 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:17.255415 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.255421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.255423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.255801 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.256114 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.256115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.256115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.256276 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.256777 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:17.256784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:17.256938 (+ 154us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.257046 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.257305 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":233}
W20260923 22:10:17.258342 16892 leader_election.cc:341] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763): Not found: Tablet not found: bffd6be7273945598129683960acaf9c
W20260923 22:10:17.258441 16892 leader_election.cc:341] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601): Not found: Tablet not found: bffd6be7273945598129683960acaf9c
I20260923 22:10:17.258477 16892 leader_election.cc:302] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a974b8fb65114f9889164c24543f8d76; no voters: 61f8947b4bab481494a1bcefda29cec3, 711362bc8ff54e03922d2fac51a3ffe0
I20260923 22:10:17.258653 17276 raft_consensus.cc:2726] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:10:17.945384 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.945177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:51453)
0923 22:10:17.945308 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.945339 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
I20260923 22:10:18.026404 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.026220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:50955)
0923 22:10:18.026331 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.026365 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260923 22:10:18.147902 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.147718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:56027)
0923 22:10:18.147832 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.147865 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260923 22:10:18.946086 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.945884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:37543)
0923 22:10:18.945999 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.946029 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260923 22:10:18.962690 17284 raft_consensus.cc:482] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:18.962800 17284 raft_consensus.cc:504] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:18.962977 17284 leader_election.cc:288] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763), 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601)
W20260923 22:10:18.963382 16892 leader_election.cc:341] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763): Not found: Tablet not found: bffd6be7273945598129683960acaf9c
W20260923 22:10:18.963470 16892 leader_election.cc:341] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601): Not found: Tablet not found: bffd6be7273945598129683960acaf9c
I20260923 22:10:18.963528 16892 leader_election.cc:302] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a974b8fb65114f9889164c24543f8d76; no voters: 61f8947b4bab481494a1bcefda29cec3, 711362bc8ff54e03922d2fac51a3ffe0
I20260923 22:10:18.963640 17284 raft_consensus.cc:2726] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:10:19.027052 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.026856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:51219)
0923 22:10:19.026980 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:19.027014 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260923 22:10:19.148561 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.148357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:44975)
0923 22:10:19.148485 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:19.148522 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260923 22:10:19.946887 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.946631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:34057)
0923 22:10:19.946793 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:19.946840 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260923 22:10:20.027771 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:20.027520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:51269)
0923 22:10:20.027701 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:20.027734 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":118,"threads_started":1}
I20260923 22:10:20.149302 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:20.149096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:37305)
0923 22:10:20.149226 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:20.149261 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260923 22:10:20.831708 17290 raft_consensus.cc:482] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:20.831802 17290 raft_consensus.cc:504] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:20.831992 17290 leader_election.cc:288] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763), 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601)
W20260923 22:10:20.832396 16892 leader_election.cc:341] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763): Not found: Tablet not found: bffd6be7273945598129683960acaf9c
W20260923 22:10:20.832473 16892 leader_election.cc:341] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601): Not found: Tablet not found: bffd6be7273945598129683960acaf9c
I20260923 22:10:20.832490 16892 leader_election.cc:302] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a974b8fb65114f9889164c24543f8d76; no voters: 61f8947b4bab481494a1bcefda29cec3, 711362bc8ff54e03922d2fac51a3ffe0
I20260923 22:10:20.832592 17290 raft_consensus.cc:2726] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:10:20.947818 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:20.947525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:40949)
0923 22:10:20.947736 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:20.947773 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":144,"threads_started":1}
I20260923 22:10:21.028440 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:21.028229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:60355)
0923 22:10:21.028371 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:21.028404 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260923 22:10:21.150125 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:21.149916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:54163)
0923 22:10:21.150049 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:21.150084 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260923 22:10:21.948489 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:21.948261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:58787)
0923 22:10:21.948406 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:21.948442 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
I20260923 22:10:22.029078 17295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:22.028879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:51125)
0923 22:10:22.029008 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:22.029041 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260923 22:10:22.150815 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:22.150591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:45929)
0923 22:10:22.150736 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:22.150772 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":63,"threads_started":1}
I20260923 22:10:22.949199 17297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:22.948986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:53399)
0923 22:10:22.949119 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:22.949155 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260923 22:10:23.029893 17298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:23.029689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:54547)
0923 22:10:23.029822 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:23.029852 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260923 22:10:23.151710 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:23.151435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:49485)
0923 22:10:23.151632 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:23.151668 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":66,"threads_started":1}
I20260923 22:10:23.949878 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:23.949672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:33379)
0923 22:10:23.949794 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:23.949829 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260923 22:10:24.030539 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:24.030321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:51413)
0923 22:10:24.030457 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:24.030494 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":59,"threads_started":1}
I20260923 22:10:24.152318 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:24.152142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:48383)
0923 22:10:24.152252 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:24.152280 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260923 22:10:24.950592 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:24.950366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:49987)
0923 22:10:24.950510 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:24.950545 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260923 22:10:25.031294 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:25.031075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:48679)
0923 22:10:25.031215 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:25.031251 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260923 22:10:25.152999 17305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:25.152795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:41111)
0923 22:10:25.152919 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:25.152957 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260923 22:10:25.712209 17306 raft_consensus.cc:482] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:25.712306 17306 raft_consensus.cc:504] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f8947b4bab481494a1bcefda29cec3" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39763 } } peers { permanent_uuid: "711362bc8ff54e03922d2fac51a3ffe0" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37601 } } peers { permanent_uuid: "a974b8fb65114f9889164c24543f8d76" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36263 } }
I20260923 22:10:25.712502 17306 leader_election.cc:288] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763), 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601)
W20260923 22:10:25.713002 16892 leader_election.cc:341] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61f8947b4bab481494a1bcefda29cec3 (127.1.244.195:39763): Not found: Tablet not found: bffd6be7273945598129683960acaf9c
W20260923 22:10:25.713083 16892 leader_election.cc:341] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 711362bc8ff54e03922d2fac51a3ffe0 (127.1.244.194:37601): Not found: Tablet not found: bffd6be7273945598129683960acaf9c
I20260923 22:10:25.713104 16892 leader_election.cc:302] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a974b8fb65114f9889164c24543f8d76; no voters: 61f8947b4bab481494a1bcefda29cec3, 711362bc8ff54e03922d2fac51a3ffe0
I20260923 22:10:25.713187 17306 raft_consensus.cc:2726] T bffd6be7273945598129683960acaf9c P a974b8fb65114f9889164c24543f8d76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:10:25.951254 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:25.951059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:59855)
0923 22:10:25.951173 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:25.951209 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260923 22:10:26.032147 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:26.031872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:40559)
0923 22:10:26.032022 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:26.032070 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260923 22:10:26.153657 17309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:26.153440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:38343)
0923 22:10:26.153562 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:26.153596 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260923 22:10:26.952035 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:26.951784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:51877)
0923 22:10:26.951953 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:26.951990 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260923 22:10:27.032827 17311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:27.032613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:35667)
0923 22:10:27.032735 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:27.032771 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260923 22:10:27.154270 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:27.154081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:46853)
0923 22:10:27.154190 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:27.154225 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260923 22:10:27.952719 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:27.952505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:46777)
0923 22:10:27.952637 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:27.952673 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260923 22:10:28.033486 17314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:28.033279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:43679)
0923 22:10:28.033411 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:28.033445 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260923 22:10:28.154947 17315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:28.154749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:34623)
0923 22:10:28.154873 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:28.154902 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260923 22:10:28.953378 17316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:28.953173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:46873)
0923 22:10:28.953286 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:28.953316 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260923 22:10:29.034147 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:29.033936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:46803)
0923 22:10:29.034065 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:29.034102 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":57,"threads_started":1}
I20260923 22:10:29.155711 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:29.155469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:40127)
0923 22:10:29.155635 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:29.155665 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":65,"threads_started":1}
I20260923 22:10:29.954110 17319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:29.953884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:56609)
0923 22:10:29.954027 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:29.954065 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260923 22:10:30.034868 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:30.034657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:33341)
0923 22:10:30.034780 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:30.034817 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260923 22:10:30.156304 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:30.156104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:42095)
0923 22:10:30.156230 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:30.156262 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
I20260923 22:10:30.954891 17322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:30.954688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.193:57161)
0923 22:10:30.954814 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:30.954848 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260923 22:10:31.035619 17323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:31.035408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.194:36663)
0923 22:10:31.035526 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:31.035577 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260923 22:10:31.156818 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 16844
I20260923 22:10:31.157572 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:31.156877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:43429 (local address 127.1.244.195:46279)
0923 22:10:31.157475 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:31.157515 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":66,"threads_started":1}
I20260923 22:10:31.161854 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 17006
I20260923 22:10:31.166103 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 17140
2026-09-23T22:10:31Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18489 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:10:31 dist-test-slave-b5jz krb5kdc[17327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:10:31 dist-test-slave-b5jz krb5kdc[17327](info): set up 2 sockets
Sep 23 22:10:31 dist-test-slave-b5jz krb5kdc[17327](info): commencing operation
krb5kdc: starting...
W20260923 22:10:32.203498 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:10:32 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:10:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:10:32.314368 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46817
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:32.394025 17340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:32.394253 17340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:32.394289 17340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:32.395432 17340 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:10:32.395506 17340 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:10:32.395527 17340 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:10:32.395546 17340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:32.395605 17340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:32.395633 17340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:32.395661 17340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:32.395684 17340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:32.397217 17340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:32.397493 17340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:32.397722 17340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:32.402647 17345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:32.402659 17346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:32.402693 17340 server_base.cc:1034] running on GCE node
W20260923 22:10:32.402690 17348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:32.403049 17340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:32.403337 17340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:32.404479 17340 hybrid_clock.cc:648] HybridClock initialized: now 1790201432404468 us; error 36 us; skew 500 ppm
Sep 23 22:10:32 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:32.406817 17340 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:10:32.407181 17340 webserver.cc:460] Webserver started at http://127.1.244.254:42371/ using document root <none> and password file <none>
I20260923 22:10:32.407368 17340 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:32.407408 17340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:32.407493 17340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:32.408531 17340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "fe94742a759f4865aa70f609d9bda0b1"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "634eb2f46a2d47a0c9c04488b4673e2f"
server_key_iv: "9dae82691524dc449fe19878562cebf1"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.408891 17340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "fe94742a759f4865aa70f609d9bda0b1"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "634eb2f46a2d47a0c9c04488b4673e2f"
server_key_iv: "9dae82691524dc449fe19878562cebf1"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.410133 17340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:10:32.410801 17355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.410957 17340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:32.411046 17340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "fe94742a759f4865aa70f609d9bda0b1"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "634eb2f46a2d47a0c9c04488b4673e2f"
server_key_iv: "9dae82691524dc449fe19878562cebf1"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.411100 17340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:32.425354 17340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:32.427620 17340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:32.427739 17340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:32.431455 17340 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46817
I20260923 22:10:32.431531 17407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46817 every 8 connection(s)
I20260923 22:10:32.431900 17340 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:32.432787 17408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:32.435045 17408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Bootstrap starting.
I20260923 22:10:32.435722 17408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:32.436033 17408 log.cc:824] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:32.436699 17408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: No bootstrap required, opened a new log
I20260923 22:10:32.438273 17408 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:10:32.438408 17408 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:32.438431 17408 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe94742a759f4865aa70f609d9bda0b1, State: Initialized, Role: FOLLOWER
I20260923 22:10:32.438604 17408 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:10:32.438671 17408 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:32.438692 17408 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:32.438757 17408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:32.439257 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 17340
I20260923 22:10:32.439334 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:10:32.439347 17408 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:10:32.439486 17408 leader_election.cc:302] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe94742a759f4865aa70f609d9bda0b1; no voters:
I20260923 22:10:32.439668 2003 external_mini_cluster.cc:1428] Setting key 496498de40076d8ae3ea6ea29e4d1405
I20260923 22:10:32.439690 17408 leader_election.cc:288] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:10:32.439724 17411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:32.439898 17411 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 1 LEADER]: Becoming Leader. State: Replica: fe94742a759f4865aa70f609d9bda0b1, State: Running, Role: LEADER
I20260923 22:10:32.439950 17408 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:32.440055 17411 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:10:32.440807 17413 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe94742a759f4865aa70f609d9bda0b1. Latest consensus state: current_term: 1 leader_uuid: "fe94742a759f4865aa70f609d9bda0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } } }
I20260923 22:10:32.440891 17413 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:32.441222 17412 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe94742a759f4865aa70f609d9bda0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } } }
I20260923 22:10:32.441291 17412 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: This master's current role is: LEADER
W20260923 22:10:32.441924 17428 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:10:32.442000 17428 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:10:32.442055 17422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:32.442203 17422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:32.443740 17422 catalog_manager.cc:1347] Generated new cluster ID: 5202e675cba84af2bf31e1a542b00fcb
I20260923 22:10:32.443789 17422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:10:32 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:32.450269 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.440312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53722 (local address 127.1.244.254:46817)
0923 22:10:32.440617 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:32.440627 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:32.440663 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:10:32.440787 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:32.440795 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:32.440852 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:32.441816 (+ 964us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:32.441902 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.443086 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.444146 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.444557 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.444693 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.446957 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:32.446961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:32.446962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:32.446973 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:32.448716 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.449104 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.449107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.449107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.449202 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.449411 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.449412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.449412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.449654 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:32.449803 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:32.449918 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 22:10:32.450055 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":100,"threads_started":1}
I20260923 22:10:32.473536 17422 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:32.474066 17422 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:32.481251 17422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Generated new TSK 0
I20260923 22:10:32.481441 17422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:10:32.510191 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46817
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:32.592643 17434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:32.592872 17434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:32.592913 17434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:32.594075 17434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:32.594148 17434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:32.594195 17434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:10:32.595716 17434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:32.595969 17434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:32.596206 17434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:32.601317 17440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:32.601312 17439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:32.601313 17442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:32.601331 17434 server_base.cc:1034] running on GCE node
I20260923 22:10:32.601691 17434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:32.601954 17434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:32.603125 17434 hybrid_clock.cc:648] HybridClock initialized: now 1790201432603113 us; error 26 us; skew 500 ppm
Sep 23 22:10:32 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:32.605885 17434 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:10:32.606348 17434 webserver.cc:460] Webserver started at http://127.1.244.193:41853/ using document root <none> and password file <none>
I20260923 22:10:32.606570 17434 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:32.606635 17434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:32.606791 17434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:32.607901 17434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "768ee67542844d669851edb7e22752d2"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "2b391739b21c04c92ece4b9aa447c9b4"
server_key_iv: "3984a7d3fd5567ea442e690620abe742"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.608276 17434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "768ee67542844d669851edb7e22752d2"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "2b391739b21c04c92ece4b9aa447c9b4"
server_key_iv: "3984a7d3fd5567ea442e690620abe742"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.609617 17434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:10:32.611564 17449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.611768 17434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:32.611897 17434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "768ee67542844d669851edb7e22752d2"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "2b391739b21c04c92ece4b9aa447c9b4"
server_key_iv: "3984a7d3fd5567ea442e690620abe742"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.611974 17434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:32.620555 17434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:32.622825 17434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:32.622968 17434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:32.623198 17434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:32.623502 17434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:32.623557 17434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.623616 17434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:32.623646 17434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.629560 17434 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:34905
I20260923 22:10:32.629623 17562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:34905 every 8 connection(s)
I20260923 22:10:32.629946 17434 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:32.635083 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 17434
I20260923 22:10:32.635200 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:10:32.635385 2003 external_mini_cluster.cc:1428] Setting key 01133d1398362ee304e461b08e6de39e
Sep 23 22:10:32 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:32.639747 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.630730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:41639 (local address 127.1.244.254:46817)
0923 22:10:32.630771 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:32.630773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:32.630977 (+ 204us) server_negotiation.cc:407] Connection header received
0923 22:10:32.631449 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:32.631450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:32.631464 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:32.631521 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:32.632503 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.633011 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.633958 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.634177 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.634232 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.636828 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:32.636830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:32.636831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:32.636839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:32.638326 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.638737 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.638738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.638738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.638791 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.639059 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.639060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.639060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.639183 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:32.639257 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:32.639542 (+ 285us) server_negotiation.cc:299] Negotiation successful
0923 22:10:32.639658 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:10:32.639927 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.630695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:41639)
0923 22:10:32.630846 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:32.630880 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:32.631347 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:32.631630 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:32.631634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:32.632204 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:32.632390 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.632395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.633107 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.633108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.633876 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.633878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.634253 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.634254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.634349 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.634863 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:32.634871 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:32.636718 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:32.638415 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:32.638424 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:32.638431 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:32.638666 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:32.638867 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:32.638868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:32.638868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:32.638985 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:32.639260 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:32.639267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:32.639404 (+ 137us) client_negotiation.cc:769] Sending connection context
0923 22:10:32.639507 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 22:10:32.639697 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":95,"threads_started":1}
I20260923 22:10:32.640657 17563 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46817
I20260923 22:10:32.640775 17563 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:32.640957 17563 heartbeater.cc:511] Master 127.1.244.254:46817 requested a full tablet report, sending...
I20260923 22:10:32.641489 17372 ts_manager.cc:194] Registered new tserver with Master: 768ee67542844d669851edb7e22752d2 (127.1.244.193:34905)
I20260923 22:10:32.642170 17372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:41639
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:10:32.662294 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46817
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:32.742604 17568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:32.742798 17568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:32.742821 17568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:32.744014 17568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:32.744055 17568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:32.744110 17568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:10:32.745772 17568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:32.746033 17568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:32.746272 17568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:32.751183 17573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:32.751138 17576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:32.751134 17574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:32.751354 17568 server_base.cc:1034] running on GCE node
I20260923 22:10:32.751587 17568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:32.751818 17568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:32.752978 17568 hybrid_clock.cc:648] HybridClock initialized: now 1790201432752960 us; error 33 us; skew 500 ppm
Sep 23 22:10:32 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:32.755376 17568 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:10:32.755857 17568 webserver.cc:460] Webserver started at http://127.1.244.194:36657/ using document root <none> and password file <none>
I20260923 22:10:32.756102 17568 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:32.756178 17568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:32.756328 17568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:32.757340 17568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "1fcdb440cfbd401792a063cb90fdbbda"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "9a2ce0ddbb79a5f7e81521a14f1318b7"
server_key_iv: "489ef6d43df5aa698295205267c54740"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.757699 17568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "1fcdb440cfbd401792a063cb90fdbbda"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "9a2ce0ddbb79a5f7e81521a14f1318b7"
server_key_iv: "489ef6d43df5aa698295205267c54740"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.758945 17568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:10:32.759653 17583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.759825 17568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:10:32.759924 17568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "1fcdb440cfbd401792a063cb90fdbbda"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "9a2ce0ddbb79a5f7e81521a14f1318b7"
server_key_iv: "489ef6d43df5aa698295205267c54740"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.760013 17568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:32.786078 17568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:32.788362 17568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:32.788468 17568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:32.788633 17568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:32.788939 17568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:32.788970 17568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.789016 17568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:32.789043 17568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.794508 17568 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:41425
I20260923 22:10:32.794622 17696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:41425 every 8 connection(s)
I20260923 22:10:32.794891 17568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:32.797068 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 17568
I20260923 22:10:32.797186 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:10:32.797394 2003 external_mini_cluster.cc:1428] Setting key b006caf791538fddc23f0b8b6539329d
Sep 23 22:10:32 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:32.804780 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.795611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:55551 (local address 127.1.244.254:46817)
0923 22:10:32.795664 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:32.795666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:32.795848 (+ 182us) server_negotiation.cc:407] Connection header received
0923 22:10:32.796307 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:32.796308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:32.796319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:32.796367 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:32.797292 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.798023 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.799003 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.799318 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.799398 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.801781 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:32.801783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:32.801783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:32.801791 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:32.803271 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.803802 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.803803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.803804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.803866 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.804118 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.804119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.804119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.804250 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:32.804326 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:32.804608 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 22:10:32.804692 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:10:32.804970 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.795607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:55551)
0923 22:10:32.795750 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:32.795784 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:32.796225 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:32.796417 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:32.796420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:32.796991 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:32.797201 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.797207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.798153 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.798154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.798914 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.798916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.799417 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.799418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.799597 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.800078 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:32.800089 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:32.801688 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:32.803383 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:32.803390 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:32.803392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:32.803716 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:32.803936 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:32.803937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:32.803937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:32.804045 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:32.804317 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:32.804323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:32.804475 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 22:10:32.804575 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 22:10:32.804728 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
I20260923 22:10:32.805517 17697 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46817
I20260923 22:10:32.805612 17697 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:32.805783 17697 heartbeater.cc:511] Master 127.1.244.254:46817 requested a full tablet report, sending...
I20260923 22:10:32.806155 17372 ts_manager.cc:194] Registered new tserver with Master: 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425)
I20260923 22:10:32.806532 17372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:55551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:10:32.820470 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46817
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:32.902303 17702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:32.902510 17702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:32.902536 17702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:32.903720 17702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:32.903759 17702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:32.903784 17702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:10:32.905097 17702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:32.905267 17702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:32.905426 17702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:32.910125 17708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:32.910125 17707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:32.910262 17702 server_base.cc:1034] running on GCE node
W20260923 22:10:32.910120 17710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:32.910526 17702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:32.910772 17702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:32.911914 17702 hybrid_clock.cc:648] HybridClock initialized: now 1790201432911893 us; error 39 us; skew 500 ppm
Sep 23 22:10:32 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:32.914518 17702 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:10:32.914925 17702 webserver.cc:460] Webserver started at http://127.1.244.195:41805/ using document root <none> and password file <none>
I20260923 22:10:32.915161 17702 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:32.915205 17702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:32.915351 17702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:32.916409 17702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "bfebe5c542ff4107a3710f453d1de1b0"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "e1b162fd3d575cd3499de4d714da48b8"
server_key_iv: "47ce56c846cd9d1194f6ea04327fd19d"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.916759 17702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "bfebe5c542ff4107a3710f453d1de1b0"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "e1b162fd3d575cd3499de4d714da48b8"
server_key_iv: "47ce56c846cd9d1194f6ea04327fd19d"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.917976 17702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:10:32.918745 17717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.918937 17702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:32.919023 17702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "bfebe5c542ff4107a3710f453d1de1b0"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "e1b162fd3d575cd3499de4d714da48b8"
server_key_iv: "47ce56c846cd9d1194f6ea04327fd19d"
server_key_version: "encryptionkey@0"
I20260923 22:10:32.919075 17702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:32.936452 17702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:32.938724 17702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:32.938884 17702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:32.939101 17702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:32.939443 17702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:32.939491 17702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.939514 17702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:32.939570 17702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:32.945525 17702 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:40863
I20260923 22:10:32.945583 17830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:40863 every 8 connection(s)
I20260923 22:10:32.945870 17702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:10:32 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:32.954782 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 17702
I20260923 22:10:32.954798 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.946612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:58391 (local address 127.1.244.254:46817)
0923 22:10:32.946675 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:32.946677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:32.946851 (+ 174us) server_negotiation.cc:407] Connection header received
0923 22:10:32.947354 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:32.947356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:32.947368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:32.947424 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:32.948419 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.948948 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.949987 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.950198 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.950284 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.952341 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:32.952343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:32.952343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:32.952349 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:32.953528 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.953894 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.953895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.953895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.953977 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.954196 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.954197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.954197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.954298 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:32.954348 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:32.954571 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 22:10:32.954691 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:10:32.954815 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.946570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:58391)
0923 22:10:32.946728 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:32.946771 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:32.947252 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:32.947495 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:32.947499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:32.948114 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:32.948316 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.948321 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.949065 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.949066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.949872 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.949874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.950293 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.950295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.950418 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.950838 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:32.950846 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:32.952256 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:32.953598 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:32.953604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:32.953605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:32.953824 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:32.954039 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:32.954039 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:32.954040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:32.954141 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:32.954351 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:32.954355 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:32.954461 (+ 106us) client_negotiation.cc:769] Sending connection context
0923 22:10:32.954538 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:10:32.954650 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260923 22:10:32.954877 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:10:32.955073 2003 external_mini_cluster.cc:1428] Setting key cb9b48d7177d76f963b7cefd3ef06292
I20260923 22:10:32.955332 17831 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46817
I20260923 22:10:32.955441 17831 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:32.955687 17831 heartbeater.cc:511] Master 127.1.244.254:46817 requested a full tablet report, sending...
I20260923 22:10:32.956068 17372 ts_manager.cc:194] Registered new tserver with Master: bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863)
I20260923 22:10:32.956465 17372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:58391
I20260923 22:10:32.967020 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:32.973699 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.968348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.1.244.254:46817)
0923 22:10:32.968390 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:32.968391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:32.968512 (+ 121us) server_negotiation.cc:407] Connection header received
0923 22:10:32.968564 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:32.968565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:32.968573 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:32.968622 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:32.969138 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.969597 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.970286 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.970473 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.970527 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.971717 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:32.971718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:32.971718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:32.971724 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:32.972804 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.973100 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.973100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.973101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.973148 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:32.973301 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:32.973301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:32.973302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:32.973424 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:32.973493 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:32.973576 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:10:32.973641 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:10:32.975071 17372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:32.976750 17372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:32.982851 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.978700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:34905 (local address 127.0.0.1:34372)
0923 22:10:32.978829 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:32.978867 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:32.978925 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:32.979245 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:32.979247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:32.979249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:32.979315 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.979316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.980660 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.980660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.981733 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.981736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.982463 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.982463 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.982551 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.982559 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 22:10:32.982689 (+ 130us) client_negotiation.cc:240] Negotiation successful
0923 22:10:32.982753 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":8,"thread_start_us":86,"threads_started":1}
I20260923 22:10:32.982868 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.978842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34372 (local address 127.1.244.193:34905)
0923 22:10:32.979072 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:32.979077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:32.979089 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:10:32.979132 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:32.979134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:32.979151 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:32.979228 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:32.979398 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.980532 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.981855 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.982340 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.982377 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.982412 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:32.982719 (+ 307us) server_negotiation.cc:299] Negotiation successful
0923 22:10:32.982771 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260923 22:10:32.982851 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.978701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41425 (local address 127.0.0.1:48204)
0923 22:10:32.978874 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:32.978885 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:32.978939 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:32.979592 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:32.979593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:32.979595 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:32.979644 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.979645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.980891 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.980892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.981717 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.981719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.982611 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.982612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.982677 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.982680 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:32.982730 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:10:32.982759 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260923 22:10:32.983144 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.979069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.1.244.194:41425)
0923 22:10:32.979371 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:32.979375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:32.979385 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:32.979442 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:32.979444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:32.979459 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:32.979719 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:32.979796 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.980784 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.981797 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.982523 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.982599 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.982652 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:32.983039 (+ 387us) server_negotiation.cc:299] Negotiation successful
0923 22:10:32.983075 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":78,"threads_started":1}
I20260923 22:10:32.983460 17842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.978923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40863 (local address 127.0.0.1:35858)
0923 22:10:32.979081 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:32.979090 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:32.979137 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:32.979465 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:32.979465 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:32.979468 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:32.979505 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.979507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.980433 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.980433 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.981798 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:32.981800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.982706 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.982707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:32.982766 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.982770 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:32.982823 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:10:32.982864 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":46,"threads_started":1}
I20260923 22:10:32.983594 17844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.979072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.1.244.195:40863)
0923 22:10:32.979255 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:32.979259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:32.979269 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:32.979303 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:32.979305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:32.979319 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:32.979393 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:32.979653 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.980342 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.981862 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:32.982526 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:32.982599 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:32.982652 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:32.983442 (+ 790us) server_negotiation.cc:299] Negotiation successful
0923 22:10:32.983492 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260923 22:10:32.983820 17631 tablet_service.cc:1467] Processing CreateTablet for tablet 592717312d704b77b1fe8470280b644e (DEFAULT_TABLE table=test-table [id=52cbf73244814a4fbfc0bd392501ac70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:32.983810 17497 tablet_service.cc:1467] Processing CreateTablet for tablet 592717312d704b77b1fe8470280b644e (DEFAULT_TABLE table=test-table [id=52cbf73244814a4fbfc0bd392501ac70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:32.984143 17497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592717312d704b77b1fe8470280b644e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:32.984161 17631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592717312d704b77b1fe8470280b644e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:32.984217 17765 tablet_service.cc:1467] Processing CreateTablet for tablet 592717312d704b77b1fe8470280b644e (DEFAULT_TABLE table=test-table [id=52cbf73244814a4fbfc0bd392501ac70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:32.984480 17765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592717312d704b77b1fe8470280b644e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:32.986625 17845 tablet_bootstrap.cc:492] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Bootstrap starting.
I20260923 22:10:32.987241 17845 tablet_bootstrap.cc:654] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:32.987574 17846 tablet_bootstrap.cc:492] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: Bootstrap starting.
I20260923 22:10:32.987771 17847 tablet_bootstrap.cc:492] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: Bootstrap starting.
I20260923 22:10:32.987921 17845 log.cc:824] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:32.988303 17846 tablet_bootstrap.cc:654] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:32.988421 17847 tablet_bootstrap.cc:654] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:32.988729 17845 tablet_bootstrap.cc:492] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: No bootstrap required, opened a new log
I20260923 22:10:32.988816 17845 ts_tablet_manager.cc:1397] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:10:32.988893 17846 log.cc:824] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:32.989009 17847 log.cc:824] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:32.989681 17846 tablet_bootstrap.cc:492] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: No bootstrap required, opened a new log
I20260923 22:10:32.989696 17847 tablet_bootstrap.cc:492] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: No bootstrap required, opened a new log
I20260923 22:10:32.989753 17846 ts_tablet_manager.cc:1397] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:32.989773 17847 ts_tablet_manager.cc:1397] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:32.990823 17845 raft_consensus.cc:348] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:32.990999 17845 raft_consensus.cc:374] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:32.991050 17845 raft_consensus.cc:729] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768ee67542844d669851edb7e22752d2, State: Initialized, Role: FOLLOWER
I20260923 22:10:32.991200 17845 consensus_queue.cc:260] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:32.991331 17847 raft_consensus.cc:348] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:32.991484 17845 ts_tablet_manager.cc:1428] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:10:32.991523 17847 raft_consensus.cc:374] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:32.991597 17847 raft_consensus.cc:729] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fcdb440cfbd401792a063cb90fdbbda, State: Initialized, Role: FOLLOWER
I20260923 22:10:32.991619 17563 heartbeater.cc:503] Master 127.1.244.254:46817 was elected leader, sending a full tablet report...
I20260923 22:10:32.991732 17847 consensus_queue.cc:260] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:32.991727 17846 raft_consensus.cc:348] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:32.991894 17846 raft_consensus.cc:374] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:32.991945 17846 raft_consensus.cc:729] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfebe5c542ff4107a3710f453d1de1b0, State: Initialized, Role: FOLLOWER
I20260923 22:10:32.992030 17847 ts_tablet_manager.cc:1428] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:32.992058 17846 consensus_queue.cc:260] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:32.992164 17697 heartbeater.cc:503] Master 127.1.244.254:46817 was elected leader, sending a full tablet report...
I20260923 22:10:32.992353 17846 ts_tablet_manager.cc:1428] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:10:32.992388 17831 heartbeater.cc:503] Master 127.1.244.254:46817 was elected leader, sending a full tablet report...
W20260923 22:10:33.045425 17698 tablet.cc:2369] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:33.056159 17852 raft_consensus.cc:482] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:33.056308 17852 raft_consensus.cc:504] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:33.056718 17852 leader_election.cc:288] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 768ee67542844d669851edb7e22752d2 (127.1.244.193:34905), bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863)
I20260923 22:10:33.060557 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:33.057056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:48571 (local address 127.1.244.193:34905)
0923 22:10:33.057113 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:33.057115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:33.057179 (+ 64us) server_negotiation.cc:407] Connection header received
0923 22:10:33.057272 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:33.057273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:33.057282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:33.057334 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:33.057484 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.058167 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.059661 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.060222 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.060297 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:33.060304 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:33.060454 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:10:33.060490 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:10:33.060573 17844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:33.057012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:37089 (local address 127.1.244.195:40863)
0923 22:10:33.057112 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:33.057115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:33.057123 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:33.057150 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:33.057152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:33.057162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:33.057219 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:33.057417 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.058163 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.059661 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.060222 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.060296 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:33.060304 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:33.060454 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:10:33.060490 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260923 22:10:33.060581 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:33.056885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40863 (local address 127.1.244.194:37089)
0923 22:10:33.056960 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:33.056971 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:33.057031 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:33.057241 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:33.057242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:33.057244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:33.057340 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:33.057342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.058246 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.058246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:33.059577 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:33.059580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.060304 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.060305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:33.060381 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:33.060382 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:10:33.060437 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:10:33.060484 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":61}
I20260923 22:10:33.060588 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:33.056885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:34905 (local address 127.1.244.194:48571)
0923 22:10:33.057112 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:33.057123 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:33.057185 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:33.057350 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:33.057351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:33.057352 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:33.057415 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:33.057417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.058246 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.058246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:33.059577 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:33.059580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.060305 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.060305 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:33.060377 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:33.060381 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:33.060437 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:10:33.060484 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":151,"thread_start_us":119,"threads_started":1}
I20260923 22:10:33.060921 17785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592717312d704b77b1fe8470280b644e" candidate_uuid: "1fcdb440cfbd401792a063cb90fdbbda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfebe5c542ff4107a3710f453d1de1b0" is_pre_election: true
I20260923 22:10:33.060985 17517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592717312d704b77b1fe8470280b644e" candidate_uuid: "1fcdb440cfbd401792a063cb90fdbbda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768ee67542844d669851edb7e22752d2" is_pre_election: true
I20260923 22:10:33.061076 17785 raft_consensus.cc:2454] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fcdb440cfbd401792a063cb90fdbbda in term 0.
I20260923 22:10:33.061151 17517 raft_consensus.cc:2454] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fcdb440cfbd401792a063cb90fdbbda in term 0.
I20260923 22:10:33.061283 17587 leader_election.cc:302] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fcdb440cfbd401792a063cb90fdbbda, bfebe5c542ff4107a3710f453d1de1b0; no voters:
I20260923 22:10:33.061424 17852 raft_consensus.cc:2781] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:33.061488 17852 raft_consensus.cc:482] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:33.061520 17852 raft_consensus.cc:3037] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:33.062281 17852 raft_consensus.cc:504] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:33.062419 17852 leader_election.cc:288] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [CANDIDATE]: Term 1 election: Requested vote from peers 768ee67542844d669851edb7e22752d2 (127.1.244.193:34905), bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863)
I20260923 22:10:33.062567 17785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592717312d704b77b1fe8470280b644e" candidate_uuid: "1fcdb440cfbd401792a063cb90fdbbda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfebe5c542ff4107a3710f453d1de1b0"
I20260923 22:10:33.062595 17517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592717312d704b77b1fe8470280b644e" candidate_uuid: "1fcdb440cfbd401792a063cb90fdbbda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768ee67542844d669851edb7e22752d2"
I20260923 22:10:33.062662 17785 raft_consensus.cc:3037] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:33.062671 17517 raft_consensus.cc:3037] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:33.063347 17785 raft_consensus.cc:2454] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fcdb440cfbd401792a063cb90fdbbda in term 1.
I20260923 22:10:33.063448 17517 raft_consensus.cc:2454] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fcdb440cfbd401792a063cb90fdbbda in term 1.
I20260923 22:10:33.063584 17587 leader_election.cc:302] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fcdb440cfbd401792a063cb90fdbbda, bfebe5c542ff4107a3710f453d1de1b0; no voters:
I20260923 22:10:33.063658 17852 raft_consensus.cc:2781] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:33.063836 17852 raft_consensus.cc:686] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 1 LEADER]: Becoming Leader. State: Replica: 1fcdb440cfbd401792a063cb90fdbbda, State: Running, Role: LEADER
I20260923 22:10:33.063979 17852 consensus_queue.cc:237] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:33.064797 17372 catalog_manager.cc:5495] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194). New cstate: current_term: 1 leader_uuid: "1fcdb440cfbd401792a063cb90fdbbda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } health_report { overall_health: UNKNOWN } } }
I20260923 22:10:33.079322 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:33.077311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.1.244.194:41425)
0923 22:10:33.077364 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:33.077366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:33.077373 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:10:33.077444 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:33.077445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:33.077459 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:33.077516 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:33.077611 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.078053 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.078713 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:33.078878 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:33.078925 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:33.079038 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:33.079132 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:33.079189 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:33.079219 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 22:10:33.079234 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:10:33.080974 17785 raft_consensus.cc:1261] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [term 1 FOLLOWER]: Refusing update from remote peer 1fcdb440cfbd401792a063cb90fdbbda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:33.080991 17517 raft_consensus.cc:1261] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Refusing update from remote peer 1fcdb440cfbd401792a063cb90fdbbda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:33.081351 17855 consensus_queue.cc:1035] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:33.081497 17852 consensus_queue.cc:1035] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [LEADER]: Connected to new peer: Peer: permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:33.086499 17370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:33.086612 17370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:33.087354 17370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Sending DeleteTablet for 3 replicas of tablet 592717312d704b77b1fe8470280b644e
I20260923 22:10:33.087661 17497 tablet_service.cc:1514] Processing DeleteTablet for tablet 592717312d704b77b1fe8470280b644e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:33 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:34372
I20260923 22:10:33.087746 17631 tablet_service.cc:1514] Processing DeleteTablet for tablet 592717312d704b77b1fe8470280b644e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:33 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:48204
I20260923 22:10:33.087884 17868 tablet_replica.cc:330] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: stopping tablet replica
I20260923 22:10:33.087954 17765 tablet_service.cc:1514] Processing DeleteTablet for tablet 592717312d704b77b1fe8470280b644e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:33 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:35858
I20260923 22:10:33.087983 17868 raft_consensus.cc:2229] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:33.087993 17869 tablet_replica.cc:330] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: stopping tablet replica
I20260923 22:10:33.088099 17868 raft_consensus.cc:2258] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:33.088141 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 17434
I20260923 22:10:33.088251 17869 raft_consensus.cc:2229] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:10:33.088402 17869 raft_consensus.cc:2258] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:33.088873 17869 ts_tablet_manager.cc:1905] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:33.090852 17869 ts_tablet_manager.cc:1918] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:33.091004 17869 log.cc:1196] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/592717312d704b77b1fe8470280b644e
I20260923 22:10:33.091361 17869 ts_tablet_manager.cc:1939] T 592717312d704b77b1fe8470280b644e P 1fcdb440cfbd401792a063cb90fdbbda: Deleting consensus metadata
I20260923 22:10:33.091871 17359 catalog_manager.cc:4841] TS 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425): tablet 592717312d704b77b1fe8470280b644e (table test-table [id=52cbf73244814a4fbfc0bd392501ac70]) successfully deleted
W20260923 22:10:33.094165 17358 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:34905 (error 108)
I20260923 22:10:33.094337 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 17568
I20260923 22:10:33.094621 17842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:33.094510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:34905 (local address 127.0.0.1:34376)
0923 22:10:33.094552 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:33.094582 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:34905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 22:10:33.094712 17358 catalog_manager.cc:4584] TS 768ee67542844d669851edb7e22752d2 (127.1.244.193:34905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 592717312d704b77b1fe8470280b644e: Network error: Client connection negotiation failed: client connection to 127.1.244.193:34905: connect: Connection refused (error 111)
I20260923 22:10:33.095748 17870 tablet_replica.cc:330] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: stopping tablet replica
I20260923 22:10:33.095868 17870 raft_consensus.cc:2229] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:33.095969 17870 raft_consensus.cc:2258] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:33.096405 17870 ts_tablet_manager.cc:1905] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:33.097977 17870 ts_tablet_manager.cc:1918] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:33.098044 17870 log.cc:1196] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/592717312d704b77b1fe8470280b644e
I20260923 22:10:33.098296 17870 ts_tablet_manager.cc:1939] T 592717312d704b77b1fe8470280b644e P bfebe5c542ff4107a3710f453d1de1b0: Deleting consensus metadata
I20260923 22:10:33.098680 17359 catalog_manager.cc:4841] TS bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863): tablet 592717312d704b77b1fe8470280b644e (table test-table [id=52cbf73244814a4fbfc0bd392501ac70]) successfully deleted
I20260923 22:10:33.099036 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 17702
I20260923 22:10:33.104070 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 17340
I20260923 22:10:53.109644 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46817
--webserver_interface=127.1.244.254
--webserver_port=42371
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:53.192719 17872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:53.192934 17872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:53.192955 17872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:53.194123 17872 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:10:53.194165 17872 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:10:53.194180 17872 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:10:53.194197 17872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:53.194252 17872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:53.194271 17872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:53.194294 17872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:53.194326 17872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:53.195905 17872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=42371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:53.196187 17872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:53.196435 17872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:53.205672 17878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:53.205646 17879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:53.205646 17881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:53.205638 17872 server_base.cc:1034] running on GCE node
I20260923 22:10:53.206197 17872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:53.206533 17872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:53.207693 17872 hybrid_clock.cc:648] HybridClock initialized: now 1790201453207683 us; error 45 us; skew 500 ppm
Sep 23 22:10:53 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201453, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:53.210708 17872 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:10:53.211222 17872 webserver.cc:460] Webserver started at http://127.1.244.254:42371/ using document root <none> and password file <none>
I20260923 22:10:53.211424 17872 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:53.211479 17872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:53.213033 17872 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:53.213765 17888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.213974 17872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:53.214087 17872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "fe94742a759f4865aa70f609d9bda0b1"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "634eb2f46a2d47a0c9c04488b4673e2f"
server_key_iv: "9dae82691524dc449fe19878562cebf1"
server_key_version: "encryptionkey@0"
I20260923 22:10:53.214514 17872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:53.231899 17872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:53.234376 17872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:53.234546 17872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:53.238965 17872 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46817
I20260923 22:10:53.239017 17940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46817 every 8 connection(s)
I20260923 22:10:53.239452 17872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:53.240958 17941 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:10:53.241654 17941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Bootstrap starting.
I20260923 22:10:53.245396 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 17872
I20260923 22:10:53.245731 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:34905
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:46817
--webserver_port=41853
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:53.250658 17941 log.cc:824] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:53.252780 17941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:53.253170 17941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Bootstrap complete.
I20260923 22:10:53.255404 17941 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:10:53.255684 17941 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe94742a759f4865aa70f609d9bda0b1, State: Initialized, Role: FOLLOWER
I20260923 22:10:53.255894 17941 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:10:53.255972 17941 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:53.256027 17941 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:53.256078 17941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:10:53.257128 17941 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:10:53.257241 17941 leader_election.cc:302] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe94742a759f4865aa70f609d9bda0b1; no voters:
I20260923 22:10:53.257414 17941 leader_election.cc:288] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:10:53.257449 17945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:10:53.257611 17945 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 2 LEADER]: Becoming Leader. State: Replica: fe94742a759f4865aa70f609d9bda0b1, State: Running, Role: LEADER
I20260923 22:10:53.257681 17941 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:53.257754 17945 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:10:53.258039 17945 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe94742a759f4865aa70f609d9bda0b1. Latest consensus state: current_term: 2 leader_uuid: "fe94742a759f4865aa70f609d9bda0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } } }
I20260923 22:10:53.258042 17946 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fe94742a759f4865aa70f609d9bda0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } } }
I20260923 22:10:53.258337 17946 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:53.258245 17945 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:53.258560 17954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:53.259524 17954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:53.259709 17954 catalog_manager.cc:1259] Loaded cluster ID: 5202e675cba84af2bf31e1a542b00fcb
I20260923 22:10:53.259922 17954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:53.260737 17954 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:53.260983 17954 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Loaded TSK: 0
I20260923 22:10:53.269424 17954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Generated new TSK 1
I20260923 22:10:53.269523 17954 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:10:53.341243 17943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:53.341485 17943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:53.341523 17943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:53.342684 17943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:53.342751 17943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:53.342796 17943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:10:53.344359 17943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:34905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=41853
--tserver_master_addrs=127.1.244.254:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:53.344630 17943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:53.344871 17943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:10:53.349964 17943 server_base.cc:1034] running on GCE node
W20260923 22:10:53.349917 17969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:53.350087 17971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:53.350176 17968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:53.350410 17943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:53.350697 17943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:53.351845 17943 hybrid_clock.cc:648] HybridClock initialized: now 1790201453351824 us; error 48 us; skew 500 ppm
Sep 23 22:10:53 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201453, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:53.354558 17943 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:10:53.355093 17943 webserver.cc:460] Webserver started at http://127.1.244.193:41853/ using document root <none> and password file <none>
I20260923 22:10:53.355329 17943 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:53.355443 17943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:53.356884 17943 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:53.357594 17978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.357780 17943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:53.357872 17943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "768ee67542844d669851edb7e22752d2"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "2b391739b21c04c92ece4b9aa447c9b4"
server_key_iv: "3984a7d3fd5567ea442e690620abe742"
server_key_version: "encryptionkey@0"
I20260923 22:10:53.358327 17943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:53.368311 17943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:53.370427 17943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:53.370589 17943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:53.370834 17943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:53.371387 17985 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:10:53.373451 17943 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:10:53.373514 17943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:10:53.373555 17943 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:10:53.374153 17943 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:10:53.374208 17943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:53.374280 17985 tablet_bootstrap.cc:492] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Bootstrap starting.
I20260923 22:10:53.381557 17943 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:34905
I20260923 22:10:53.381633 18092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:34905 every 8 connection(s)
I20260923 22:10:53.382032 17943 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:53.385380 17985 log.cc:824] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Log is configured to *not* fsync() on all Append() calls
Sep 23 22:10:53 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201453, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:53.389736 17985 tablet_bootstrap.cc:492] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:53.390074 17985 tablet_bootstrap.cc:492] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Bootstrap complete.
I20260923 22:10:53.391202 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 17943
I20260923 22:10:53.391237 17985 ts_tablet_manager.cc:1397] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.008s
I20260923 22:10:53.391496 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:41425
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:46817
--webserver_port=36657
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:53.392436 17985 raft_consensus.cc:348] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:53.392843 17985 raft_consensus.cc:729] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768ee67542844d669851edb7e22752d2, State: Initialized, Role: FOLLOWER
I20260923 22:10:53.393049 17985 consensus_queue.cc:260] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } }
I20260923 22:10:53.393335 17985 ts_tablet_manager.cc:1428] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260923 22:10:53.395978 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.382763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:45877)
0923 22:10:53.383004 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:53.383049 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:53.383477 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:53.384057 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:53.384061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:53.384329 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:53.384514 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.384519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.385860 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.385862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.387110 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.387114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.387815 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.387817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.387950 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.388474 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:53.388482 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:53.389916 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:53.393946 (+ 4030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:53.393952 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:53.393954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:53.394272 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:53.394597 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:53.394598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:53.394598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:53.394731 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:53.395223 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:53.395231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:53.395394 (+ 163us) client_negotiation.cc:769] Sending connection context
0923 22:10:53.395516 (+ 122us) client_negotiation.cc:240] Negotiation successful
0923 22:10:53.395722 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":87,"threads_started":1}
I20260923 22:10:53.396638 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.382821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:45877 (local address 127.1.244.254:46817)
0923 22:10:53.383697 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.383704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.383733 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:10:53.383831 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.383836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.383874 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.384014 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:53.384636 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.385691 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.387213 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.387698 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.387814 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.390041 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:53.390045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:53.390045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:53.390058 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:53.393845 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.394378 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.394381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.394382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.394512 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.394803 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.394804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.394804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.395121 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:53.396120 (+ 999us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.396202 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.396359 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":78,"threads_started":1}
I20260923 22:10:53.397248 18093 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46817
I20260923 22:10:53.397351 18093 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:53.397550 18093 heartbeater.cc:511] Master 127.1.244.254:46817 requested a full tablet report, sending...
I20260923 22:10:53.398386 17905 ts_manager.cc:194] Registered new tserver with Master: 768ee67542844d669851edb7e22752d2 (127.1.244.193:34905)
I20260923 22:10:53.400100 17905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:45877
I20260923 22:10:53.401191 18093 heartbeater.cc:503] Master 127.1.244.254:46817 was elected leader, sending a full tablet report...
Sep 23 22:10:53 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201453, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260923 22:10:53.409361 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.399759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:34905 (local address 127.0.0.1:58792)
0923 22:10:53.400123 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:53.400173 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:53.400263 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:53.400627 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:53.400632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:53.401022 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:53.401083 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.401086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.402333 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.402334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.403349 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.403352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.404008 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.404010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.404090 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.404594 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:53.404599 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:53.406184 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:53.408137 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:53.408144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:53.408146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:53.408410 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:53.408631 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:53.408632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:53.408632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:53.408697 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:53.409011 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:53.409017 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:53.409067 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 22:10:53.409152 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 22:10:53.409262 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":93,"threads_started":1}
I20260923 22:10:53.409394 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.399883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.1.244.193:34905)
0923 22:10:53.400054 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.400059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.400351 (+ 292us) server_negotiation.cc:407] Connection header received
0923 22:10:53.400453 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.400456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.400477 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.400551 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:53.401178 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.402238 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.403438 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.403930 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.403999 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.406278 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:53.406285 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:53.406285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:53.406292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:53.408056 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.408497 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.408499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.408500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.408561 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.408762 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.408763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.408763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.408942 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:53.409023 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.409202 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.409300 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260923 22:10:53.410028 18027 tablet_service.cc:1514] Processing DeleteTablet for tablet 592717312d704b77b1fe8470280b644e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:33 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:58792
I20260923 22:10:53.410609 18102 tablet_replica.cc:330] stopping tablet replica
I20260923 22:10:53.410723 18102 raft_consensus.cc:2229] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:53.410810 18102 raft_consensus.cc:2258] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:53.411136 18102 ts_tablet_manager.cc:1905] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:53.413546 18102 ts_tablet_manager.cc:1918] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:53.413641 18102 log.cc:1196] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal/wals/592717312d704b77b1fe8470280b644e
I20260923 22:10:53.413856 18102 ts_tablet_manager.cc:1939] T 592717312d704b77b1fe8470280b644e P 768ee67542844d669851edb7e22752d2: Deleting consensus metadata
I20260923 22:10:53.414327 17891 catalog_manager.cc:4841] TS 768ee67542844d669851edb7e22752d2 (127.1.244.193:34905): tablet 592717312d704b77b1fe8470280b644e (table test-table [id=52cbf73244814a4fbfc0bd392501ac70]) successfully deleted
W20260923 22:10:53.478797 18098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:53.479005 18098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:53.479028 18098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:53.480192 18098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:53.480233 18098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:53.480259 18098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:10:53.481626 18098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:41425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=36657
--tserver_master_addrs=127.1.244.254:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:53.481818 18098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:53.482120 18098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:53.487015 18107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:53.487036 18108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:53.487027 18110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:53.487247 18098 server_base.cc:1034] running on GCE node
I20260923 22:10:53.487462 18098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:53.487763 18098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:53.488942 18098 hybrid_clock.cc:648] HybridClock initialized: now 1790201453488933 us; error 28 us; skew 500 ppm
Sep 23 22:10:53 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201453, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:53.491690 18098 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:10:53.492128 18098 webserver.cc:460] Webserver started at http://127.1.244.194:36657/ using document root <none> and password file <none>
I20260923 22:10:53.492393 18098 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:53.492473 18098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:53.493844 18098 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:53.494405 18117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.494603 18098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.494702 18098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "1fcdb440cfbd401792a063cb90fdbbda"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "9a2ce0ddbb79a5f7e81521a14f1318b7"
server_key_iv: "489ef6d43df5aa698295205267c54740"
server_key_version: "encryptionkey@0"
I20260923 22:10:53.495070 18098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:53.506122 18098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:53.507980 18098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:53.508131 18098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:53.508343 18098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:53.508635 18098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:53.508688 18098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.508730 18098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:53.508759 18098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.514535 18098 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:41425
I20260923 22:10:53.514583 18230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:41425 every 8 connection(s)
I20260923 22:10:53.514909 18098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:53.516850 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 18098
I20260923 22:10:53.517115 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:40863
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:46817
--webserver_port=41805
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:10:53 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201453, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:53.525665 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.515630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:47317 (local address 127.1.244.254:46817)
0923 22:10:53.515693 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.515695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.515968 (+ 273us) server_negotiation.cc:407] Connection header received
0923 22:10:53.516475 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.516477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.516489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.516560 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:53.517701 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.518497 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.520028 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.520267 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.520327 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.522727 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:53.522729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:53.522730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:53.522737 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:53.524137 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.524546 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.524547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.524548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.524607 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.524858 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.524859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.524859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.525014 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:53.525094 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.525478 (+ 384us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.525572 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:10:53.525817 18233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.515658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:47317)
0923 22:10:53.515847 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:53.515882 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:53.516349 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:53.516615 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:53.516619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:53.517321 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:53.517614 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.517619 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.518596 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.518597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.519943 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.519946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.520355 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.520356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.520502 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.521043 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:53.521054 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:53.522626 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:53.524237 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:53.524244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:53.524246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:53.524467 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:53.524675 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:53.524676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:53.524676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:53.524788 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:53.525086 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:53.525092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:53.525299 (+ 207us) client_negotiation.cc:769] Sending connection context
0923 22:10:53.525435 (+ 136us) client_negotiation.cc:240] Negotiation successful
0923 22:10:53.525596 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260923 22:10:53.526242 18231 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46817
I20260923 22:10:53.526333 18231 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:53.526530 18231 heartbeater.cc:511] Master 127.1.244.254:46817 requested a full tablet report, sending...
I20260923 22:10:53.526904 17905 ts_manager.cc:194] Registered new tserver with Master: 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425)
I20260923 22:10:53.527292 17905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:47317
W20260923 22:10:53.600402 18234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:53.600620 18234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:53.600658 18234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:53.601814 18234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:53.601877 18234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:53.601922 18234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:10:53.603371 18234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:40863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=41805
--tserver_master_addrs=127.1.244.254:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:10:53.603662 18234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:53.603888 18234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:53.608754 18242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:53.608749 18239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:53.608785 18240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:53.609133 18234 server_base.cc:1034] running on GCE node
I20260923 22:10:53.609346 18234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:53.609532 18234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:53.610666 18234 hybrid_clock.cc:648] HybridClock initialized: now 1790201453610653 us; error 28 us; skew 500 ppm
Sep 23 22:10:53 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201453, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:53.613159 18234 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:10:53.613585 18234 webserver.cc:460] Webserver started at http://127.1.244.195:41805/ using document root <none> and password file <none>
I20260923 22:10:53.613821 18234 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:53.613876 18234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:53.615233 18234 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:53.615885 18249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.616079 18234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:53.616168 18234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "bfebe5c542ff4107a3710f453d1de1b0"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "e1b162fd3d575cd3499de4d714da48b8"
server_key_iv: "47ce56c846cd9d1194f6ea04327fd19d"
server_key_version: "encryptionkey@0"
I20260923 22:10:53.616508 18234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:53.624840 18234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:53.626748 18234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:53.626858 18234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:53.627023 18234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:53.627422 18234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:53.627457 18234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.627478 18234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:53.627524 18234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.633437 18234 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:40863
I20260923 22:10:53.633486 18362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:40863 every 8 connection(s)
I20260923 22:10:53.633805 18234 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:10:53 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201453, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:10:53.642632 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.634442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:53125)
0923 22:10:53.634617 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:53.634651 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:53.635114 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:53.635324 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:53.635327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:53.635955 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:53.636143 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.636148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.636813 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.636814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.637598 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.637600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.637944 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.637944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.638045 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.638479 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:53.638487 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:53.639827 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:53.641390 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:53.641394 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:53.641396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:53.641586 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:53.641831 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:53.641832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:53.641832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:53.641922 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:53.642161 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:53.642165 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:53.642276 (+ 111us) client_negotiation.cc:769] Sending connection context
0923 22:10:53.642363 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 22:10:53.642475 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":117,"threads_started":1}
I20260923 22:10:53.642678 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.634478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:53125 (local address 127.1.244.254:46817)
0923 22:10:53.634572 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.634574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.634739 (+ 165us) server_negotiation.cc:407] Connection header received
0923 22:10:53.635198 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.635200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.635212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.635265 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:53.636240 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.636734 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.637679 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.637873 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.637927 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.639914 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:53.639915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:53.639916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:53.639922 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:53.641323 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.641695 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.641696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.641696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.641761 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.641983 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.641984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.641984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.642104 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:53.642161 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.642398 (+ 237us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.642553 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 22:10:53.642681 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 18234
I20260923 22:10:53.643133 18363 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46817
I20260923 22:10:53.643229 18363 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:53.643414 18363 heartbeater.cc:511] Master 127.1.244.254:46817 requested a full tablet report, sending...
I20260923 22:10:53.643849 17905 ts_manager.cc:194] Registered new tserver with Master: bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863)
I20260923 22:10:53.644296 17905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:53125
I20260923 22:10:53.649876 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.643266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.1.244.254:46817)
0923 22:10:53.643304 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.643306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.643370 (+ 64us) server_negotiation.cc:407] Connection header received
0923 22:10:53.643465 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.643466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.643477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.643531 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:53.644283 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.644858 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.645718 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.645939 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.646022 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.647344 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:53.647345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:53.647346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:53.647353 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:53.648665 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.649065 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.649066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.649067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.649128 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.649366 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.649367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.649368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.649478 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:53.649668 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.649705 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.649778 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:10:53.650892 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:53.653541 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.651228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54060 (local address 127.1.244.254:46817)
0923 22:10:53.651267 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.651269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.651415 (+ 146us) server_negotiation.cc:407] Connection header received
0923 22:10:53.651457 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.651458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.651486 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.651600 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:53.651726 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.652178 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.652923 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.653099 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.653152 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.653310 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:53.653352 (+ 42us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0923 22:10:53.653486 (+ 134us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54060: token expired
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:10:53.653611 18096 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54060: token expired
W20260923 22:10:53.654017 17834 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.244.254:46817)
I20260923 22:10:53.662640 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.654272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.1.244.254:46817)
0923 22:10:53.654395 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.654410 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.654418 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:53.654467 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.654468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.654478 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.654559 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:53.655339 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.656194 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.657387 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.657658 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.657746 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.659517 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:53.659519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:53.659519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:53.659526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:53.661418 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.661786 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.661787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.661788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.661865 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:53.662103 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:53.662104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:53.662104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:53.662231 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:53.662322 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.662476 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.662550 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 22:10:53.664577 17905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:53.666154 17905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:53.668401 18027 tablet_service.cc:1467] Processing CreateTablet for tablet 7b40ff90f3374b93927b9630d413af34 (DEFAULT_TABLE table=test-table [id=b03390eae9cc40d7b3ef812860a716d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:53.668619 18027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b40ff90f3374b93927b9630d413af34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:53.670235 17985 tablet_bootstrap.cc:492] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Bootstrap starting.
I20260923 22:10:53.670583 17985 tablet_bootstrap.cc:654] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:53.671523 17985 tablet_bootstrap.cc:492] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: No bootstrap required, opened a new log
I20260923 22:10:53.671614 17985 ts_tablet_manager.cc:1397] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:53.671753 17985 raft_consensus.cc:348] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:10:53.671844 17985 raft_consensus.cc:374] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:53.671890 17985 raft_consensus.cc:729] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768ee67542844d669851edb7e22752d2, State: Initialized, Role: FOLLOWER
I20260923 22:10:53.671943 17985 consensus_queue.cc:260] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:10:53.672025 17985 ts_tablet_manager.cc:1428] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:53.672458 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.668095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40863 (local address 127.0.0.1:49422)
0923 22:10:53.668132 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:53.668142 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:53.668176 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:53.668647 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:53.668648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:53.668650 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:53.668699 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.668700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.669897 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.669898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.671000 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.671004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.672142 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.672144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.672265 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.672287 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 22:10:53.672345 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:10:53.672395 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260923 22:10:53.672741 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.668235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.1.244.195:40863)
0923 22:10:53.668444 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.668449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.668461 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:10:53.668501 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.668504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.668520 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.668633 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:53.668771 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.669815 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.671105 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.672052 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.672132 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.672207 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.672623 (+ 416us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.672668 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":121,"threads_started":1}
I20260923 22:10:53.673367 18297 tablet_service.cc:1467] Processing CreateTablet for tablet 7b40ff90f3374b93927b9630d413af34 (DEFAULT_TABLE table=test-table [id=b03390eae9cc40d7b3ef812860a716d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:53.673697 18297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b40ff90f3374b93927b9630d413af34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:53.674206 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.668194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41425 (local address 127.0.0.1:52666)
0923 22:10:53.668320 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:53.668331 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:53.668385 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:53.670627 (+ 2242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:53.670628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:53.670631 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:53.670678 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.670680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.671846 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.671848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.673091 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.673093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.673975 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.673976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.674038 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.674043 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:10:53.674100 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:10:53.674137 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260923 22:10:53.674228 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.670298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.1.244.194:41425)
0923 22:10:53.670457 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.670462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.670472 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:53.670509 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.670511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.670533 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.670630 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:53.670728 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.671662 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.673178 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.673896 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.673975 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.674029 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.674113 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.674152 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260923 22:10:53.674815 18165 tablet_service.cc:1467] Processing CreateTablet for tablet 7b40ff90f3374b93927b9630d413af34 (DEFAULT_TABLE table=test-table [id=b03390eae9cc40d7b3ef812860a716d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:53.675097 18165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b40ff90f3374b93927b9630d413af34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:53.676761 18372 tablet_bootstrap.cc:492] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: Bootstrap starting.
I20260923 22:10:53.677289 18372 tablet_bootstrap.cc:654] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:53.677807 18372 log.cc:824] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:53.677933 18373 tablet_bootstrap.cc:492] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: Bootstrap starting.
I20260923 22:10:53.678524 18372 tablet_bootstrap.cc:492] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: No bootstrap required, opened a new log
I20260923 22:10:53.678546 18373 tablet_bootstrap.cc:654] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:53.678609 18372 ts_tablet_manager.cc:1397] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:53.679009 18373 log.cc:824] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:53.679710 18373 tablet_bootstrap.cc:492] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: No bootstrap required, opened a new log
I20260923 22:10:53.679764 18373 ts_tablet_manager.cc:1397] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:53.680133 18372 raft_consensus.cc:348] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:10:53.680290 18372 raft_consensus.cc:374] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:53.680334 18372 raft_consensus.cc:729] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfebe5c542ff4107a3710f453d1de1b0, State: Initialized, Role: FOLLOWER
I20260923 22:10:53.680472 18372 consensus_queue.cc:260] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:10:53.680744 18372 ts_tablet_manager.cc:1428] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:53.680804 18363 heartbeater.cc:503] Master 127.1.244.254:46817 was elected leader, sending a full tablet report...
I20260923 22:10:53.681605 18373 raft_consensus.cc:348] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:10:53.681780 18373 raft_consensus.cc:374] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:53.681824 18373 raft_consensus.cc:729] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fcdb440cfbd401792a063cb90fdbbda, State: Initialized, Role: FOLLOWER
I20260923 22:10:53.681936 18373 consensus_queue.cc:260] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:10:53.682215 18373 ts_tablet_manager.cc:1428] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:10:53.682276 18231 heartbeater.cc:503] Master 127.1.244.254:46817 was elected leader, sending a full tablet report...
I20260923 22:10:53.740530 18377 raft_consensus.cc:482] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:53.740733 18377 raft_consensus.cc:504] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:10:53.741158 18377 leader_election.cc:288] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863), 768ee67542844d669851edb7e22752d2 (127.1.244.193:34905)
I20260923 22:10:53.744253 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.741498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:42349 (local address 127.1.244.195:40863)
0923 22:10:53.741543 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.741545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.741557 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:10:53.741593 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.741594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.741600 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.741682 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:53.741896 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.742476 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.743428 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.743885 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.743974 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.743983 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.744146 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.744186 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:10:53.744253 18233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.741361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40863 (local address 127.1.244.194:42349)
0923 22:10:53.741411 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:53.741425 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:53.741473 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:53.741731 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:53.741732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:53.741734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:53.741811 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.741813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.742571 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.742572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.743318 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.743320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.743971 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.743972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.744056 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.744059 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:53.744124 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:10:53.744170 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260923 22:10:53.744614 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.741439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:42601 (local address 127.1.244.193:34905)
0923 22:10:53.741509 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.741511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.741781 (+ 270us) server_negotiation.cc:407] Connection header received
0923 22:10:53.741900 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.741902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.741928 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.742018 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:53.742207 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.742823 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.743823 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.744235 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.744324 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.744401 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.744484 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.744538 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:10:53.744632 18317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b40ff90f3374b93927b9630d413af34" candidate_uuid: "1fcdb440cfbd401792a063cb90fdbbda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfebe5c542ff4107a3710f453d1de1b0" is_pre_election: true
I20260923 22:10:53.744836 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.741406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:34905 (local address 127.1.244.194:42601)
0923 22:10:53.741706 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:53.741717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:53.741808 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:53.742022 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:53.742023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:53.742025 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:53.742120 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.742122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.742909 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.742910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.743729 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:53.743732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.744329 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.744330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:53.744393 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.744396 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:53.744721 (+ 325us) client_negotiation.cc:240] Negotiation successful
0923 22:10:53.744768 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":108,"threads_started":1}
I20260923 22:10:53.744838 18317 raft_consensus.cc:2454] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fcdb440cfbd401792a063cb90fdbbda in term 0.
I20260923 22:10:53.745079 18121 leader_election.cc:302] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fcdb440cfbd401792a063cb90fdbbda, bfebe5c542ff4107a3710f453d1de1b0; no voters:
I20260923 22:10:53.745117 18047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b40ff90f3374b93927b9630d413af34" candidate_uuid: "1fcdb440cfbd401792a063cb90fdbbda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768ee67542844d669851edb7e22752d2" is_pre_election: true
I20260923 22:10:53.745220 18377 raft_consensus.cc:2781] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:53.745277 18047 raft_consensus.cc:2454] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fcdb440cfbd401792a063cb90fdbbda in term 0.
I20260923 22:10:53.745313 18377 raft_consensus.cc:482] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:53.745347 18377 raft_consensus.cc:3037] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:53.746178 18377 raft_consensus.cc:504] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:10:53.746323 18377 leader_election.cc:288] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [CANDIDATE]: Term 1 election: Requested vote from peers bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863), 768ee67542844d669851edb7e22752d2 (127.1.244.193:34905)
I20260923 22:10:53.746456 18317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b40ff90f3374b93927b9630d413af34" candidate_uuid: "1fcdb440cfbd401792a063cb90fdbbda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfebe5c542ff4107a3710f453d1de1b0"
I20260923 22:10:53.746466 18047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b40ff90f3374b93927b9630d413af34" candidate_uuid: "1fcdb440cfbd401792a063cb90fdbbda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768ee67542844d669851edb7e22752d2"
I20260923 22:10:53.746546 18047 raft_consensus.cc:3037] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:53.746559 18317 raft_consensus.cc:3037] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:53.747370 18317 raft_consensus.cc:2454] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fcdb440cfbd401792a063cb90fdbbda in term 1.
I20260923 22:10:53.747407 18047 raft_consensus.cc:2454] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fcdb440cfbd401792a063cb90fdbbda in term 1.
I20260923 22:10:53.747745 18121 leader_election.cc:302] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fcdb440cfbd401792a063cb90fdbbda, bfebe5c542ff4107a3710f453d1de1b0; no voters:
I20260923 22:10:53.748016 18377 raft_consensus.cc:2781] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:53.748188 18377 raft_consensus.cc:686] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 1 LEADER]: Becoming Leader. State: Replica: 1fcdb440cfbd401792a063cb90fdbbda, State: Running, Role: LEADER
I20260923 22:10:53.748291 18377 consensus_queue.cc:237] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:10:53.749388 17905 catalog_manager.cc:5495] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194). New cstate: current_term: 1 leader_uuid: "1fcdb440cfbd401792a063cb90fdbbda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } health_report { overall_health: UNKNOWN } } }
W20260923 22:10:53.765398 18232 tablet.cc:2369] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:53.766518 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.762937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.1.244.194:41425)
0923 22:10:53.763102 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:53.763110 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:53.763121 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:10:53.763160 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:53.763161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:53.763181 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:53.763268 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:53.763417 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.764387 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.765556 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:53.765886 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:53.765944 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:53.766128 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:53.766223 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:53.766280 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:53.766376 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:10:53.766409 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260923 22:10:53.768416 18047 raft_consensus.cc:1261] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Refusing update from remote peer 1fcdb440cfbd401792a063cb90fdbbda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:53.768456 18317 raft_consensus.cc:1261] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [term 1 FOLLOWER]: Refusing update from remote peer 1fcdb440cfbd401792a063cb90fdbbda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:53.768749 18377 consensus_queue.cc:1035] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [LEADER]: Connected to new peer: Peer: permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:53.768908 18379 consensus_queue.cc:1035] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:53.771755 18383 mvcc.cc:205] Tried to move back new op lower bound from 7332665154632515584 to 7332665154553794560. Current Snapshot: MvccSnapshot[applied={T|T < 7332665154632515584}]
I20260923 22:10:53.772137 18382 mvcc.cc:205] Tried to move back new op lower bound from 7332665154632515584 to 7332665154553794560. Current Snapshot: MvccSnapshot[applied={T|T < 7332665154632515584}]
I20260923 22:10:53.773540 17904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:53.773630 17904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:53.774358 17904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Sending DeleteTablet for 3 replicas of tablet 7b40ff90f3374b93927b9630d413af34
I20260923 22:10:53.774643 18297 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b40ff90f3374b93927b9630d413af34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:53 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:49422
I20260923 22:10:53.774693 18027 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b40ff90f3374b93927b9630d413af34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:53 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:58792
I20260923 22:10:53.774682 18165 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b40ff90f3374b93927b9630d413af34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:53 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:52666
I20260923 22:10:53.774770 18102 tablet_replica.cc:330] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: stopping tablet replica
I20260923 22:10:53.774832 18102 raft_consensus.cc:2229] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:53.774887 18102 raft_consensus.cc:2258] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:53.774919 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 17943
I20260923 22:10:53.775763 18391 tablet_replica.cc:330] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: stopping tablet replica
I20260923 22:10:53.775884 18391 raft_consensus.cc:2229] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:10:53.775982 18391 raft_consensus.cc:2258] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:53.776335 18391 ts_tablet_manager.cc:1905] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:53.777943 18391 ts_tablet_manager.cc:1918] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:53.778040 18391 log.cc:1196] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/7b40ff90f3374b93927b9630d413af34
I20260923 22:10:53.778307 18391 ts_tablet_manager.cc:1939] T 7b40ff90f3374b93927b9630d413af34 P 1fcdb440cfbd401792a063cb90fdbbda: Deleting consensus metadata
I20260923 22:10:53.778692 17892 catalog_manager.cc:4841] TS 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425): tablet 7b40ff90f3374b93927b9630d413af34 (table test-table [id=b03390eae9cc40d7b3ef812860a716d0]) successfully deleted
W20260923 22:10:53.782243 17891 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:34905 (error 108)
I20260923 22:10:53.782508 18390 tablet_replica.cc:330] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: stopping tablet replica
I20260923 22:10:53.782629 18390 raft_consensus.cc:2229] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:53.782763 18390 raft_consensus.cc:2258] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:53.782977 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 18098
I20260923 22:10:53.783900 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:53.783763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:34905 (local address 127.0.0.1:58800)
0923 22:10:53.783811 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:53.783857 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:34905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:10:53.784015 17891 catalog_manager.cc:4584] TS 768ee67542844d669851edb7e22752d2 (127.1.244.193:34905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b40ff90f3374b93927b9630d413af34: Network error: Client connection negotiation failed: client connection to 127.1.244.193:34905: connect: Connection refused (error 111)
I20260923 22:10:53.784652 18390 ts_tablet_manager.cc:1905] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:53.786088 18390 ts_tablet_manager.cc:1918] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:53.786180 18390 log.cc:1196] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/7b40ff90f3374b93927b9630d413af34
I20260923 22:10:53.786487 18390 ts_tablet_manager.cc:1939] T 7b40ff90f3374b93927b9630d413af34 P bfebe5c542ff4107a3710f453d1de1b0: Deleting consensus metadata
I20260923 22:10:53.786899 17892 catalog_manager.cc:4841] TS bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863): tablet 7b40ff90f3374b93927b9630d413af34 (table test-table [id=b03390eae9cc40d7b3ef812860a716d0]) successfully deleted
I20260923 22:10:53.788519 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 18234
I20260923 22:10:53.795343 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 17872
I20260923 22:11:13.803233 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46817
--webserver_interface=127.1.244.254
--webserver_port=42371
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:13.886282 18393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:13.886571 18393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:13.886597 18393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:13.887862 18393 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:11:13.887900 18393 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:11:13.887913 18393 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:11:13.887925 18393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:13.887969 18393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:13.887993 18393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:13.888016 18393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:13.888036 18393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:13.889981 18393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=42371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:13.890267 18393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:13.890514 18393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:13.895854 18399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:13.895892 18393 server_base.cc:1034] running on GCE node
W20260923 22:11:13.896150 18401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:13.895855 18398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:13.896556 18393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:13.896944 18393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:13.898121 18393 hybrid_clock.cc:648] HybridClock initialized: now 1790201473898109 us; error 55 us; skew 500 ppm
Sep 23 22:11:13 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201473, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:13.901292 18393 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:11:13.901840 18393 webserver.cc:460] Webserver started at http://127.1.244.254:42371/ using document root <none> and password file <none>
I20260923 22:11:13.902122 18393 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:13.902186 18393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:13.904112 18393 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:11:13.904901 18408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:13.905081 18393 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:11:13.905167 18393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "fe94742a759f4865aa70f609d9bda0b1"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "634eb2f46a2d47a0c9c04488b4673e2f"
server_key_iv: "9dae82691524dc449fe19878562cebf1"
server_key_version: "encryptionkey@0"
I20260923 22:11:13.905507 18393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:13.919113 18393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:13.921774 18393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:13.921936 18393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:13.927299 18393 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46817
I20260923 22:11:13.927340 18460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46817 every 8 connection(s)
I20260923 22:11:13.927832 18393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:13.928232 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 18393
I20260923 22:11:13.928571 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:34905
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:46817
--webserver_port=41853
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:11:13.930037 18461 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:11:13.930739 18461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Bootstrap starting.
I20260923 22:11:13.939186 18461 log.cc:824] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:13.941483 18461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:11:13.941817 18461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: Bootstrap complete.
I20260923 22:11:13.944159 18461 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:11:13.944380 18461 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe94742a759f4865aa70f609d9bda0b1, State: Initialized, Role: FOLLOWER
I20260923 22:11:13.944581 18461 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:11:13.944648 18461 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:13.944700 18461 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:13.944743 18461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:11:13.946142 18461 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:11:13.946235 18461 leader_election.cc:302] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe94742a759f4865aa70f609d9bda0b1; no voters:
I20260923 22:11:13.946414 18461 leader_election.cc:288] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 22:11:13.946455 18465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:11:13.946579 18465 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [term 3 LEADER]: Becoming Leader. State: Replica: fe94742a759f4865aa70f609d9bda0b1, State: Running, Role: LEADER
I20260923 22:11:13.946667 18465 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } }
I20260923 22:11:13.946694 18461 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:13.946946 18466 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fe94742a759f4865aa70f609d9bda0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } } }
I20260923 22:11:13.947036 18466 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:13.947171 18467 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe94742a759f4865aa70f609d9bda0b1. Latest consensus state: current_term: 3 leader_uuid: "fe94742a759f4865aa70f609d9bda0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe94742a759f4865aa70f609d9bda0b1" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46817 } } }
I20260923 22:11:13.947232 18467 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:13.948822 18481 catalog_manager.cc:1259] Loaded cluster ID: 5202e675cba84af2bf31e1a542b00fcb
I20260923 22:11:13.948868 18481 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: loading cluster ID for follower catalog manager: success
I20260923 22:11:13.950006 18481 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fe94742a759f4865aa70f609d9bda0b1: acquiring CA information for follower catalog manager: success
F20260923 22:11:13.950369 18481 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f07a5b989ad google::LogMessage::Fail() at ??:0
@ 0x7f07a5b9b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f07a5b984de google::LogMessage::Flush() at ??:0
@ 0x7f07a5b99a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f07a8305104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f07adaf56b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f07adaf5989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f07adaf5e4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f07adaf88c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f07a7a5b954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f07a88bf6db start_thread at ??:0
@ 0x7f07a469661f clone at ??:0
W20260923 22:11:14.025542 18463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:14.025764 18463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:14.025802 18463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:14.026932 18463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:14.026996 18463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:14.027042 18463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:11:14.028594 18463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:34905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=41853
--tserver_master_addrs=127.1.244.254:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:14.028847 18463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:14.029111 18463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:14.034085 18500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:14.034158 18502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:14.034233 18463 server_base.cc:1034] running on GCE node
W20260923 22:11:14.034406 18499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:14.034648 18463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:14.034940 18463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:14.036100 18463 hybrid_clock.cc:648] HybridClock initialized: now 1790201474036075 us; error 42 us; skew 500 ppm
Sep 23 22:11:14 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201474, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:14.038518 18463 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:11:14.038980 18463 webserver.cc:460] Webserver started at http://127.1.244.193:41853/ using document root <none> and password file <none>
I20260923 22:11:14.039203 18463 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:14.039273 18463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:14.040715 18463 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:14.041430 18509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:14.041633 18463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:14.041733 18463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "768ee67542844d669851edb7e22752d2"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "2b391739b21c04c92ece4b9aa447c9b4"
server_key_iv: "3984a7d3fd5567ea442e690620abe742"
server_key_version: "encryptionkey@0"
I20260923 22:11:14.042151 18463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:14.060731 18463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:14.064332 18463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:14.064505 18463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:14.064780 18463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:14.065284 18516 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:11:14.066747 18463 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:11:14.066811 18463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:11:14.066867 18463 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:11:14.067835 18463 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:11:14.067886 18463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:14.067962 18516 tablet_bootstrap.cc:492] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Bootstrap starting.
I20260923 22:11:14.075366 18463 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:34905
I20260923 22:11:14.075613 18623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:34905 every 8 connection(s)
I20260923 22:11:14.075945 18463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:14.078603 18516 log.cc:824] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:14.080734 18516 tablet_bootstrap.cc:492] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:11:14.081028 18516 tablet_bootstrap.cc:492] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Bootstrap complete.
I20260923 22:11:14.081987 18516 ts_tablet_manager.cc:1397] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260923 22:11:14.082963 18516 raft_consensus.cc:348] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:11:14.083201 18516 raft_consensus.cc:729] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768ee67542844d669851edb7e22752d2, State: Initialized, Role: FOLLOWER
I20260923 22:11:14.083359 18516 consensus_queue.cc:260] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:11:14.083665 18516 ts_tablet_manager.cc:1428] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20260923 22:11:14.084220 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 18463
I20260923 22:11:14.084503 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:41425
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:46817
--webserver_port=36657
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:14.163462 18626 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:11:14.076883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:60347)
0923 22:11:14.077149 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:14.077204 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:14.077955 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:14.163245 (+ 85290us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":120,"threads_started":1}
W20260923 22:11:14.163738 18624 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:46817 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:46817: Client connection negotiation failed: client connection to 127.1.244.254:46817: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:11:14.164080 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:14.163951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:33411)
0923 22:11:14.164007 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:14.164047 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 22:11:14.164539 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:14.164361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:36597)
0923 22:11:14.164475 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:14.164512 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260923 22:11:14.164669 18624 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:11:14.166301 18629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:14.166518 18629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:14.166567 18629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:14.167881 18629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:14.167943 18629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:14.167986 18629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:11:14.169582 18629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:41425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=36657
--tserver_master_addrs=127.1.244.254:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:14.169839 18629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:14.170063 18629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:14.174834 18637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:14.174856 18634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:14.174834 18635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:14.175101 18629 server_base.cc:1034] running on GCE node
I20260923 22:11:14.175274 18629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:14.175458 18629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:14.176627 18629 hybrid_clock.cc:648] HybridClock initialized: now 1790201474176616 us; error 40 us; skew 500 ppm
Sep 23 22:11:14 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201474, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:14.178992 18629 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:11:14.179406 18629 webserver.cc:460] Webserver started at http://127.1.244.194:36657/ using document root <none> and password file <none>
I20260923 22:11:14.179678 18629 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:14.179733 18629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:14.181056 18629 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:14.181622 18644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:14.181788 18629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:14.181885 18629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "1fcdb440cfbd401792a063cb90fdbbda"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "9a2ce0ddbb79a5f7e81521a14f1318b7"
server_key_iv: "489ef6d43df5aa698295205267c54740"
server_key_version: "encryptionkey@0"
I20260923 22:11:14.182210 18629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:14.198199 18629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:14.200079 18629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:14.200233 18629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:14.200452 18629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:14.200760 18629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:14.200812 18629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:14.200855 18629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:14.200883 18629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:14.206557 18629 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:41425
I20260923 22:11:14.206656 18757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:41425 every 8 connection(s)
I20260923 22:11:14.206979 18629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:14.208086 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:14.207718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:34051)
0923 22:11:14.207875 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:14.207944 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
W20260923 22:11:14.208500 18758 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:46817 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:46817: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
I20260923 22:11:14.208758 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:14.208668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:49547)
0923 22:11:14.208715 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:14.208741 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:11:14.209048 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:14.208998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:41231)
0923 22:11:14.209020 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:14.209037 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260923 22:11:14.209157 18758 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:11:14.209642 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 18629
I20260923 22:11:14.209939 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:40863
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:46817
--webserver_port=41805
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:14.291539 18761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:14.291843 18761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:14.291880 18761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:14.293038 18761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:14.293107 18761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:14.293150 18761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:11:14.294606 18761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:40863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=41805
--tserver_master_addrs=127.1.244.254:46817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:14.294873 18761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:14.295102 18761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:14.300094 18769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:14.300093 18766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:14.300177 18767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:14.300318 18761 server_base.cc:1034] running on GCE node
I20260923 22:11:14.300549 18761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:14.300824 18761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:14.301971 18761 hybrid_clock.cc:648] HybridClock initialized: now 1790201474301960 us; error 31 us; skew 500 ppm
Sep 23 22:11:14 dist-test-slave-b5jz krb5kdc[17327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201474, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:14.304531 18761 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:11:14.304967 18761 webserver.cc:460] Webserver started at http://127.1.244.195:41805/ using document root <none> and password file <none>
I20260923 22:11:14.305191 18761 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:14.305262 18761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:14.306547 18761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:14.307128 18776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:14.307310 18761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:14.307410 18761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "bfebe5c542ff4107a3710f453d1de1b0"
format_stamp: "Formatted at 2026-09-23 22:10:32 on dist-test-slave-b5jz"
server_key: "e1b162fd3d575cd3499de4d714da48b8"
server_key_iv: "47ce56c846cd9d1194f6ea04327fd19d"
server_key_version: "encryptionkey@0"
I20260923 22:11:14.307808 18761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:14.338028 18761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:14.340124 18761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:14.340438 18761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:14.340703 18761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:14.341050 18761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:14.341109 18761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:14.341150 18761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:14.341194 18761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:14.347268 18761 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:40863
I20260923 22:11:14.347323 18889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:40863 every 8 connection(s)
I20260923 22:11:14.347683 18761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:14.348766 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:14.348345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:52935)
0923 22:11:14.348502 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:14.348624 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
W20260923 22:11:14.349192 18890 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:46817 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:46817: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
I20260923 22:11:14.349514 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:14.349421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:53609)
0923 22:11:14.349468 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:14.349496 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:11:14.349792 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:14.349734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:57599)
0923 22:11:14.349759 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:14.349779 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260923 22:11:14.349915 18890 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:11:14.354954 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 18761
I20260923 22:11:15.165444 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:15.165165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:43101)
0923 22:11:15.165335 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:15.165367 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260923 22:11:15.209671 18895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:15.209481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:57963)
0923 22:11:15.209594 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:15.209624 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260923 22:11:15.347869 18896 raft_consensus.cc:482] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:15.347988 18896 raft_consensus.cc:504] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:11:15.348336 18896 leader_election.cc:288] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863), 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425)
I20260923 22:11:15.350414 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:15.350227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:32945)
0923 22:11:15.350332 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:15.350375 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":78,"threads_started":1}
Sep 23 22:11:15 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201474, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
Sep 23 22:11:15 dist-test-slave-b5jz krb5kdc[17327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201474, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260923 22:11:15.359009 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:15.348508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40863 (local address 127.1.244.193:38503)
0923 22:11:15.348574 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:15.348588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:15.348666 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:15.349506 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:15.349509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:15.349912 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:15.350129 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:15.350132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:15.351767 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:15.351768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:15.352596 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:15.352598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:15.353177 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:15.353178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:15.353264 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:15.353832 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:15.353842 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:15.355470 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:15.357539 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:15.357546 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:15.357548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:15.357834 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:15.358099 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:15.358099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:15.358100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:15.358213 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:15.358634 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:15.358635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:15.358746 (+ 111us) client_negotiation.cc:769] Sending connection context
0923 22:11:15.358814 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:11:15.358941 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":44}
I20260923 22:11:15.359124 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:15.348626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41425 (local address 127.1.244.193:53061)
0923 22:11:15.348732 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:15.348742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:15.348812 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:15.349612 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:15.349613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:15.349913 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:15.350119 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:15.350126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:15.351801 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:15.351802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:15.352595 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:15.352598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:15.353800 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:15.353801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:15.353905 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:15.354323 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:15.354328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:15.355831 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:15.357609 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:15.357610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:15.357610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:15.357853 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:15.358107 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:15.358107 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:15.358108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:15.358218 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:15.358599 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:15.358606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:15.358751 (+ 145us) client_negotiation.cc:769] Sending connection context
0923 22:11:15.358845 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:11:15.359027 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":111,"thread_start_us":43,"threads_started":1}
I20260923 22:11:15.359177 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:15.348839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53061 (local address 127.1.244.194:41425)
0923 22:11:15.348968 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:15.348976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:15.349000 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:11:15.349425 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:15.349428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:15.349479 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:15.349623 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:15.350287 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:15.351670 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:15.353264 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:15.353721 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:15.353825 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:15.355951 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:15.355955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:15.355955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:15.355966 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:15.357507 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:15.357935 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:15.357939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:15.357939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:15.358028 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:15.358287 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:15.358288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:15.358288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:15.358518 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:15.358606 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:15.358927 (+ 321us) server_negotiation.cc:299] Negotiation successful
0923 22:11:15.359068 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":48,"threads_started":1}
I20260923 22:11:15.359541 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:15.348656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:38503 (local address 127.1.244.195:40863)
0923 22:11:15.348787 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:15.348798 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:15.348818 (+ 20us) server_negotiation.cc:407] Connection header received
0923 22:11:15.349264 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:15.349266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:15.349324 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:15.349448 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:15.350245 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:15.351673 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:15.352688 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:15.353100 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:15.353191 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:15.355596 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:15.355601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:15.355601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:15.355613 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:15.357433 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:15.357920 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:15.357923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:15.357924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:15.358020 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:15.358286 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:15.358287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:15.358287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:15.358540 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:15.359161 (+ 621us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:15.359248 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:11:15.359413 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
W20260923 22:11:15.360357 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
W20260923 22:11:15.360459 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
I20260923 22:11:15.360515 18513 leader_election.cc:302] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 768ee67542844d669851edb7e22752d2; no voters: 1fcdb440cfbd401792a063cb90fdbbda, bfebe5c542ff4107a3710f453d1de1b0
I20260923 22:11:15.360641 18896 raft_consensus.cc:2726] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:16.166188 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.165958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:36187)
0923 22:11:16.166106 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.166144 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260923 22:11:16.210342 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.210111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:46713)
0923 22:11:16.210262 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.210293 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260923 22:11:16.351116 18903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.350908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:44563)
0923 22:11:16.351043 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.351076 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260923 22:11:17.167060 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.166738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:39569)
0923 22:11:17.166913 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:17.166955 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260923 22:11:17.210948 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.210719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:48909)
0923 22:11:17.210877 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:17.210907 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
I20260923 22:11:17.351752 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.351533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:54637)
0923 22:11:17.351674 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:17.351712 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260923 22:11:17.523643 18907 raft_consensus.cc:482] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:17.523744 18907 raft_consensus.cc:504] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:11:17.523943 18907 leader_election.cc:288] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863), 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425)
W20260923 22:11:17.524389 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
W20260923 22:11:17.524475 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
I20260923 22:11:17.524511 18513 leader_election.cc:302] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 768ee67542844d669851edb7e22752d2; no voters: 1fcdb440cfbd401792a063cb90fdbbda, bfebe5c542ff4107a3710f453d1de1b0
I20260923 22:11:17.524595 18907 raft_consensus.cc:2726] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:18.167865 18908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:18.167630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:42319)
0923 22:11:18.167780 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:18.167814 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260923 22:11:18.211580 18909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:18.211352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:48051)
0923 22:11:18.211476 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:18.211503 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260923 22:11:18.352409 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:18.352174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:51557)
0923 22:11:18.352309 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:18.352352 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260923 22:11:19.168670 18911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:19.168385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:59111)
0923 22:11:19.168551 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:19.168592 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260923 22:11:19.212266 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:19.211987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:44367)
0923 22:11:19.212170 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:19.212215 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260923 22:11:19.353044 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:19.352835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:60467)
0923 22:11:19.352963 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:19.353001 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260923 22:11:19.941817 18914 raft_consensus.cc:482] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:19.941927 18914 raft_consensus.cc:504] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:11:19.942127 18914 leader_election.cc:288] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863), 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425)
W20260923 22:11:19.942613 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
W20260923 22:11:19.942706 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
I20260923 22:11:19.942739 18513 leader_election.cc:302] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 768ee67542844d669851edb7e22752d2; no voters: 1fcdb440cfbd401792a063cb90fdbbda, bfebe5c542ff4107a3710f453d1de1b0
I20260923 22:11:19.942824 18914 raft_consensus.cc:2726] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:20.169430 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:20.169126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:58805)
0923 22:11:20.169311 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:20.169366 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":120,"threads_started":1}
I20260923 22:11:20.212894 18916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:20.212695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:44695)
0923 22:11:20.212813 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:20.212852 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260923 22:11:20.353801 18917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:20.353515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:47995)
0923 22:11:20.353688 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:20.353756 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":81,"threads_started":1}
I20260923 22:11:21.170094 18918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:21.169884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:52287)
0923 22:11:21.170006 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:21.170047 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260923 22:11:21.213558 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:21.213347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:38891)
0923 22:11:21.213471 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:21.213514 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260923 22:11:21.354453 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:21.354230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:35867)
0923 22:11:21.354372 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:21.354411 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260923 22:11:22.170712 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:22.170499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:33727)
0923 22:11:22.170630 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:22.170666 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260923 22:11:22.214180 18922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:22.213964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:44791)
0923 22:11:22.214104 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:22.214139 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260923 22:11:22.355124 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:22.354911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:54841)
0923 22:11:22.355040 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:22.355080 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260923 22:11:23.171571 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:23.171238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:49873)
0923 22:11:23.171461 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:23.171507 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":61,"threads_started":1}
I20260923 22:11:23.214768 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:23.214602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:39539)
0923 22:11:23.214704 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:23.214732 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":55,"threads_started":1}
I20260923 22:11:23.349006 18926 raft_consensus.cc:482] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:23.349090 18926 raft_consensus.cc:504] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:11:23.349243 18926 leader_election.cc:288] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863), 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425)
W20260923 22:11:23.349606 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
W20260923 22:11:23.349694 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
I20260923 22:11:23.349728 18513 leader_election.cc:302] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 768ee67542844d669851edb7e22752d2; no voters: 1fcdb440cfbd401792a063cb90fdbbda, bfebe5c542ff4107a3710f453d1de1b0
I20260923 22:11:23.349833 18926 raft_consensus.cc:2726] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:23.355756 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:23.355576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:46223)
0923 22:11:23.355681 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:23.355716 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":51,"threads_started":1}
I20260923 22:11:24.172230 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:24.172023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:60125)
0923 22:11:24.172146 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:24.172182 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260923 22:11:24.215426 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:24.215206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:47241)
0923 22:11:24.215338 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:24.215379 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260923 22:11:24.356328 18930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:24.356142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:50001)
0923 22:11:24.356253 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:24.356287 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260923 22:11:25.172828 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:25.172632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:42973)
0923 22:11:25.172751 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:25.172781 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260923 22:11:25.216058 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:25.215872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:43979)
0923 22:11:25.215984 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:25.216018 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260923 22:11:25.356999 18933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:25.356759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:59333)
0923 22:11:25.356891 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:25.356936 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260923 22:11:25.617765 18934 raft_consensus.cc:482] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:25.617862 18934 raft_consensus.cc:504] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfebe5c542ff4107a3710f453d1de1b0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40863 } } peers { permanent_uuid: "1fcdb440cfbd401792a063cb90fdbbda" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41425 } } peers { permanent_uuid: "768ee67542844d669851edb7e22752d2" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34905 } }
I20260923 22:11:25.618068 18934 leader_election.cc:288] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863), 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425)
W20260923 22:11:25.618468 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fcdb440cfbd401792a063cb90fdbbda (127.1.244.194:41425): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
W20260923 22:11:25.618556 18513 leader_election.cc:341] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bfebe5c542ff4107a3710f453d1de1b0 (127.1.244.195:40863): Not found: Tablet not found: 7b40ff90f3374b93927b9630d413af34
I20260923 22:11:25.618593 18513 leader_election.cc:302] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 768ee67542844d669851edb7e22752d2; no voters: 1fcdb440cfbd401792a063cb90fdbbda, bfebe5c542ff4107a3710f453d1de1b0
I20260923 22:11:25.618683 18934 raft_consensus.cc:2726] T 7b40ff90f3374b93927b9630d413af34 P 768ee67542844d669851edb7e22752d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:26.173456 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:26.173252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:44853)
0923 22:11:26.173374 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:26.173410 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260923 22:11:26.216625 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:26.216421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:52977)
0923 22:11:26.216551 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:26.216585 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260923 22:11:26.357625 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:26.357434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:60177)
0923 22:11:26.357542 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:26.357574 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260923 22:11:27.174322 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:27.174012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:37241)
0923 22:11:27.174232 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:27.174273 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260923 22:11:27.217288 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:27.217070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:39709)
0923 22:11:27.217200 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:27.217235 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260923 22:11:27.358310 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:27.358024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:35481)
0923 22:11:27.358223 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:27.358264 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":59,"threads_started":1}
I20260923 22:11:28.174966 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:28.174730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:38123)
0923 22:11:28.174877 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:28.174919 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
I20260923 22:11:28.218014 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:28.217787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:41747)
0923 22:11:28.217919 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:28.217956 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260923 22:11:28.359069 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:28.358852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:56777)
0923 22:11:28.358991 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:28.359027 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260923 22:11:29.175701 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:29.175482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.193:44691)
0923 22:11:29.175628 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:29.175657 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260923 22:11:29.218715 18945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:29.218500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.194:33293)
0923 22:11:29.218641 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:29.218672 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260923 22:11:29.359882 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:29.359655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46817 (local address 127.1.244.195:42071)
0923 22:11:29.359802 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:29.359839 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":59,"threads_started":1}
I20260923 22:11:29.363636 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 18463
I20260923 22:11:29.368427 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 18629
I20260923 22:11:29.372478 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 18761
2026-09-23T22:11:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58207 ms)
[----------] 30 tests from SecurityITest (232933 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:29 dist-test-slave-b5jz krb5kdc[18949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:29 dist-test-slave-b5jz krb5kdc[18949](info): set up 2 sockets
Sep 23 22:11:29 dist-test-slave-b5jz krb5kdc[18949](info): commencing operation
krb5kdc: starting...
W20260923 22:11:30.407860 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:30 dist-test-slave-b5jz krb5kdc[18949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:11:30.524326 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39565
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:39565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:30.605101 18962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:30.605306 18962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:30.605327 18962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:30.606488 18962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:30.606536 18962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:30.606550 18962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:30.606563 18962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:30.606576 18962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:30.608178 18962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:39565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:30.608431 18962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:30.608687 18962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:30.613837 18967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:30.613874 18962 server_base.cc:1034] running on GCE node
W20260923 22:11:30.613885 18968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:30.613844 18970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:30.614300 18962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:30.614603 18962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:30.615765 18962 hybrid_clock.cc:648] HybridClock initialized: now 1790201490615748 us; error 47 us; skew 500 ppm
Sep 23 22:11:30 dist-test-slave-b5jz krb5kdc[18949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201490, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:30.618260 18962 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:11:30.618764 18962 webserver.cc:460] Webserver started at http://127.1.244.254:33925/ using document root <none> and password file <none>
I20260923 22:11:30.618996 18962 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:30.619065 18962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:30.619220 18962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:30.620342 18962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "d7cc058b3b44444dafbcb95870db879a"
format_stamp: "Formatted at 2026-09-23 22:11:30 on dist-test-slave-b5jz"
server_key: "2acab385daaff50af4323fbe94f117c1"
server_key_iv: "664647581c1cf8ecfc8a1e2832a2367a"
server_key_version: "encryptionkey@0"
I20260923 22:11:30.620707 18962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "d7cc058b3b44444dafbcb95870db879a"
format_stamp: "Formatted at 2026-09-23 22:11:30 on dist-test-slave-b5jz"
server_key: "2acab385daaff50af4323fbe94f117c1"
server_key_iv: "664647581c1cf8ecfc8a1e2832a2367a"
server_key_version: "encryptionkey@0"
I20260923 22:11:30.622043 18962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:11:30.622771 18977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:30.622974 18962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:30.623077 18962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "d7cc058b3b44444dafbcb95870db879a"
format_stamp: "Formatted at 2026-09-23 22:11:30 on dist-test-slave-b5jz"
server_key: "2acab385daaff50af4323fbe94f117c1"
server_key_iv: "664647581c1cf8ecfc8a1e2832a2367a"
server_key_version: "encryptionkey@0"
I20260923 22:11:30.623162 18962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:30.642612 18962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:30.644857 18962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:30.645044 18962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:30.649134 18962 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:39565
I20260923 22:11:30.649169 19029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:39565 every 8 connection(s)
I20260923 22:11:30.649539 18962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:30.650473 19030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:30.652832 19030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a: Bootstrap starting.
I20260923 22:11:30.653378 19030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:30.653612 19030 log.cc:824] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:30.654259 19030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a: No bootstrap required, opened a new log
I20260923 22:11:30.655933 19030 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7cc058b3b44444dafbcb95870db879a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39565 } }
I20260923 22:11:30.656071 19030 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:30.656095 19030 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7cc058b3b44444dafbcb95870db879a, State: Initialized, Role: FOLLOWER
I20260923 22:11:30.656237 19030 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7cc058b3b44444dafbcb95870db879a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39565 } }
I20260923 22:11:30.656325 19030 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:30.656350 19030 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:30.656386 19030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:30.656885 19030 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7cc058b3b44444dafbcb95870db879a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39565 } }
I20260923 22:11:30.656967 19030 leader_election.cc:302] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7cc058b3b44444dafbcb95870db879a; no voters:
I20260923 22:11:30.657089 19030 leader_election.cc:288] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:30.657157 19033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:30.657333 19033 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [term 1 LEADER]: Becoming Leader. State: Replica: d7cc058b3b44444dafbcb95870db879a, State: Running, Role: LEADER
I20260923 22:11:30.657361 19030 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:30.657433 19033 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7cc058b3b44444dafbcb95870db879a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39565 } }
I20260923 22:11:30.658027 19034 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7cc058b3b44444dafbcb95870db879a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7cc058b3b44444dafbcb95870db879a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39565 } } }
I20260923 22:11:30.658129 19034 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:30.658236 19035 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7cc058b3b44444dafbcb95870db879a. Latest consensus state: current_term: 1 leader_uuid: "d7cc058b3b44444dafbcb95870db879a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7cc058b3b44444dafbcb95870db879a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39565 } } }
I20260923 22:11:30.658382 19035 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:30.658465 19043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:30.659024 19043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:30.659256 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 18962
I20260923 22:11:30.659384 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:30.659626 2003 external_mini_cluster.cc:1428] Setting key 00e099aff085df20de181594bedb3deb
I20260923 22:11:30.661418 19043 catalog_manager.cc:1347] Generated new cluster ID: 1010b63a356c459ba23c5a08aa3199a0
I20260923 22:11:30.661473 19043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:30.669811 19043 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:30.670394 19043 catalog_manager.cc:1504] Loading token signing keys...
Sep 23 22:11:30 dist-test-slave-b5jz krb5kdc[18949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:30.678089 19043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a: Generated new TSK 0
I20260923 22:11:30.678308 19043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:30.681422 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:30.660249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.1.244.254:39565)
0923 22:11:30.669569 (+ 9320us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:30.669581 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:30.669624 (+ 43us) server_negotiation.cc:407] Connection header received
0923 22:11:30.669764 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:30.669773 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:30.669849 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:30.670023 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:30.670915 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:30.672323 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:30.673458 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:30.674078 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:30.674175 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:30.677080 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:30.677084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:30.677085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:30.677099 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:30.679399 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:30.679823 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:30.679827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:30.679827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:30.679942 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:30.680128 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:30.680129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:30.680130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:30.680428 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:30.680590 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:30.680658 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 22:11:30.680813 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9265,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:11:30.708525 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39565
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:30.825340 19056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:30.825559 19056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:30.825582 19056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:30.826807 19056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:30.826931 19056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:30.826989 19056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:11:30.828526 19056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:30.828773 19056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:30.829072 19056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:30.835059 19061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:30.835091 19064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:30.835297 19062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:30.835273 19056 server_base.cc:1034] running on GCE node
I20260923 22:11:30.835659 19056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:30.835960 19056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:30.837136 19056 hybrid_clock.cc:648] HybridClock initialized: now 1790201490837124 us; error 29 us; skew 500 ppm
Sep 23 22:11:30 dist-test-slave-b5jz krb5kdc[18949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201490, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:30.839943 19056 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:11:30.840417 19056 webserver.cc:460] Webserver started at http://127.1.244.193:38559/ using document root <none> and password file <none>
I20260923 22:11:30.840679 19056 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:30.840751 19056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:30.840936 19056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:30.842085 19056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "9ffc8af4dcdb4648878dceb03d591862"
format_stamp: "Formatted at 2026-09-23 22:11:30 on dist-test-slave-b5jz"
server_key: "576138d3b76004cf9f592d549ca51fb6"
server_key_iv: "b5b4d231f81fe78fdf82107cfd03d923"
server_key_version: "encryptionkey@0"
I20260923 22:11:30.842478 19056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "9ffc8af4dcdb4648878dceb03d591862"
format_stamp: "Formatted at 2026-09-23 22:11:30 on dist-test-slave-b5jz"
server_key: "576138d3b76004cf9f592d549ca51fb6"
server_key_iv: "b5b4d231f81fe78fdf82107cfd03d923"
server_key_version: "encryptionkey@0"
I20260923 22:11:30.843969 19056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:30.844686 19071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:30.844846 19056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:30.844945 19056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "9ffc8af4dcdb4648878dceb03d591862"
format_stamp: "Formatted at 2026-09-23 22:11:30 on dist-test-slave-b5jz"
server_key: "576138d3b76004cf9f592d549ca51fb6"
server_key_iv: "b5b4d231f81fe78fdf82107cfd03d923"
server_key_version: "encryptionkey@0"
I20260923 22:11:30.845019 19056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:30.854678 19056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:30.856920 19056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:30.857072 19056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:30.857283 19056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:30.857575 19056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:30.857626 19056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:30.857668 19056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:30.857707 19056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:30.863479 19056 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:44905
I20260923 22:11:30.863615 19184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:44905 every 8 connection(s)
I20260923 22:11:30.863886 19056 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:30.864117 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 19056
I20260923 22:11:30.864223 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:30.864449 2003 external_mini_cluster.cc:1428] Setting key 7d4b12f99d4a2ee5b573077eb68f359c
Sep 23 22:11:30 dist-test-slave-b5jz krb5kdc[18949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201490, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260923 22:11:30.874063 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:30.864650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:52741 (local address 127.1.244.254:39565)
0923 22:11:30.864703 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:30.864705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:30.865038 (+ 333us) server_negotiation.cc:407] Connection header received
0923 22:11:30.865592 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:30.865593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:30.865605 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:30.865676 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:30.866746 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:30.867234 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:30.868320 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:30.868684 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:30.868828 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:30.871183 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:30.871185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:30.871185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:30.871193 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:30.872826 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:30.873261 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:30.873263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:30.873263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:30.873333 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:30.873547 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:30.873548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:30.873548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:30.873643 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:30.873689 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:30.873915 (+ 226us) server_negotiation.cc:299] Negotiation successful
0923 22:11:30.873997 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:11:30.874342 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:30.864649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39565 (local address 127.1.244.193:52741)
0923 22:11:30.864938 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:30.864974 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:30.865494 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:30.865730 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:30.865734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:30.866335 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:30.866641 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:30.866648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:30.867327 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:30.867328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:30.868233 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:30.868236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:30.868837 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:30.868838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:30.868943 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:30.869489 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:30.869498 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:30.871084 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:30.872905 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:30.872911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:30.872913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:30.873189 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:30.873410 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:30.873410 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:30.873411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:30.873509 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:30.873692 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:30.873696 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:30.873808 (+ 112us) client_negotiation.cc:769] Sending connection context
0923 22:11:30.873909 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 22:11:30.874087 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":220,"threads_started":1}
I20260923 22:11:30.874962 19185 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39565
I20260923 22:11:30.875061 19185 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:30.875259 19185 heartbeater.cc:511] Master 127.1.244.254:39565 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
I20260923 22:11:30.875860 18994 ts_manager.cc:194] Registered new tserver with Master: 9ffc8af4dcdb4648878dceb03d591862 (127.1.244.193:44905)
I20260923 22:11:30.876482 18994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:52741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:11:30.888377 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39565
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:30.971704 19190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:30.971956 19190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:30.971997 19190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:30.973198 19190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:30.973274 19190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:30.973320 19190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:11:30.974771 19190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:30.975044 19190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:30.975286 19190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:30.980732 19196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:30.980795 19195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:30.980748 19190 server_base.cc:1034] running on GCE node
W20260923 22:11:30.980732 19198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:30.981225 19190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:30.981513 19190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:30.982664 19190 hybrid_clock.cc:648] HybridClock initialized: now 1790201490982640 us; error 40 us; skew 500 ppm
Sep 23 22:11:30 dist-test-slave-b5jz krb5kdc[18949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201490, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:30.985471 19190 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:11:30.985901 19190 webserver.cc:460] Webserver started at http://127.1.244.194:36259/ using document root <none> and password file <none>
I20260923 22:11:30.986121 19190 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:30.986167 19190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:30.986276 19190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:30.987380 19190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "18be22ee74a04cba91e3c04099c76467"
format_stamp: "Formatted at 2026-09-23 22:11:30 on dist-test-slave-b5jz"
server_key: "522495d74bd6f9b0cd80dc9646c2b9c6"
server_key_iv: "fff8b28392b389719edc03e6fb7b0bcf"
server_key_version: "encryptionkey@0"
I20260923 22:11:30.987789 19190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "18be22ee74a04cba91e3c04099c76467"
format_stamp: "Formatted at 2026-09-23 22:11:30 on dist-test-slave-b5jz"
server_key: "522495d74bd6f9b0cd80dc9646c2b9c6"
server_key_iv: "fff8b28392b389719edc03e6fb7b0bcf"
server_key_version: "encryptionkey@0"
I20260923 22:11:30.989154 19190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:30.989905 19205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:30.990094 19190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:30.990208 19190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "18be22ee74a04cba91e3c04099c76467"
format_stamp: "Formatted at 2026-09-23 22:11:30 on dist-test-slave-b5jz"
server_key: "522495d74bd6f9b0cd80dc9646c2b9c6"
server_key_iv: "fff8b28392b389719edc03e6fb7b0bcf"
server_key_version: "encryptionkey@0"
I20260923 22:11:30.990312 19190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:31.016634 19190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:31.018690 19190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:31.018883 19190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:31.019138 19190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:31.019487 19190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:31.019552 19190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.019626 19190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:31.019657 19190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.025532 19190 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:34559
I20260923 22:11:31.025579 19318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:34559 every 8 connection(s)
I20260923 22:11:31.025931 19190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:11:31 dist-test-slave-b5jz krb5kdc[18949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201490, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:31.033332 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 19190
I20260923 22:11:31.033460 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:31.033679 2003 external_mini_cluster.cc:1428] Setting key 780ebffd61fcd39ae7aaf6bc6ce893ec
I20260923 22:11:31.035813 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.026683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:40149 (local address 127.1.244.254:39565)
0923 22:11:31.026753 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.026756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.026978 (+ 222us) server_negotiation.cc:407] Connection header received
0923 22:11:31.027465 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.027466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.027476 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.027539 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:31.028551 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.029097 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.030110 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.030294 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.030351 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.032684 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:31.032685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:31.032685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:31.032693 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:31.034257 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:31.034739 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:31.034740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:31.034741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:31.034810 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:31.035067 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:31.035067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:31.035068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:31.035241 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:31.035313 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.035579 (+ 266us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.035675 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:11:31.035904 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.026659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39565 (local address 127.1.244.194:40149)
0923 22:11:31.026843 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.026885 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.027372 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.027620 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.027624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.028232 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:31.028433 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.028438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.029187 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.029188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.030032 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.030034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.030374 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.030375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.030484 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.031074 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:31.031083 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:31.032570 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:31.034333 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:31.034340 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:31.034342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:31.034657 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:31.034886 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:31.034887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:31.034887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:31.034995 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:31.035309 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:31.035313 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:31.035430 (+ 117us) client_negotiation.cc:769] Sending connection context
0923 22:11:31.035514 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 22:11:31.035705 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
I20260923 22:11:31.036506 19319 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39565
I20260923 22:11:31.036604 19319 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:31.036803 19319 heartbeater.cc:511] Master 127.1.244.254:39565 requested a full tablet report, sending...
I20260923 22:11:31.037235 18994 ts_manager.cc:194] Registered new tserver with Master: 18be22ee74a04cba91e3c04099c76467 (127.1.244.194:34559)
I20260923 22:11:31.037616 18994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:40149
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:11:31.060132 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39565
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:31.139879 19324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:31.140081 19324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:31.140106 19324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:31.141289 19324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:31.141333 19324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:31.141360 19324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:11:31.142804 19324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:31.143057 19324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:31.143286 19324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:31.148238 19329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:31.148248 19332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:31.148259 19330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:31.148491 19324 server_base.cc:1034] running on GCE node
I20260923 22:11:31.148687 19324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:31.148939 19324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:31.150110 19324 hybrid_clock.cc:648] HybridClock initialized: now 1790201491150097 us; error 28 us; skew 500 ppm
Sep 23 22:11:31 dist-test-slave-b5jz krb5kdc[18949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201491, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:31.152601 19324 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:11:31.153066 19324 webserver.cc:460] Webserver started at http://127.1.244.195:34459/ using document root <none> and password file <none>
I20260923 22:11:31.153304 19324 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:31.153376 19324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:31.153527 19324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:31.154651 19324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "bc993467a3b34ed68227d3db34239814"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "b5070623dd6328e42096e171f175b15a"
server_key_iv: "d2ba9771edefb9f16f7aa84945e32ed7"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.155016 19324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "bc993467a3b34ed68227d3db34239814"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "b5070623dd6328e42096e171f175b15a"
server_key_iv: "d2ba9771edefb9f16f7aa84945e32ed7"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.156379 19324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:11:31.157232 19339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.157420 19324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:31.157512 19324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "bc993467a3b34ed68227d3db34239814"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "b5070623dd6328e42096e171f175b15a"
server_key_iv: "d2ba9771edefb9f16f7aa84945e32ed7"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.157586 19324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:31.170441 19324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:31.172633 19324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:31.172830 19324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:31.173080 19324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:31.173383 19324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:31.173444 19324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.173482 19324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:31.173532 19324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.179318 19324 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:42789
I20260923 22:11:31.179389 19452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:42789 every 8 connection(s)
I20260923 22:11:31.179764 19324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:31.184993 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 19324
I20260923 22:11:31.185117 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:31.185313 2003 external_mini_cluster.cc:1428] Setting key 9f2d2c09f74902ce0abccb5bdb5f9b70
Sep 23 22:11:31 dist-test-slave-b5jz krb5kdc[18949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201491, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:31.189567 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.180566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:34249 (local address 127.1.244.254:39565)
0923 22:11:31.180670 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.180672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.181155 (+ 483us) server_negotiation.cc:407] Connection header received
0923 22:11:31.181522 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.181523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.181535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.181599 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:31.182531 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.183031 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.184091 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.184297 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.184377 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.186814 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:31.186816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:31.186816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:31.186824 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:31.188334 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:31.188733 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:31.188734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:31.188734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:31.188786 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:31.189020 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:31.189020 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:31.189021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:31.189120 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:31.189171 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.189429 (+ 258us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.189501 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260923 22:11:31.189759 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.180491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39565 (local address 127.1.244.195:34249)
0923 22:11:31.180700 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.180740 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.181412 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.181672 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.181675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.182226 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:31.182418 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.182423 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.183127 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.183128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.183987 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.183990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.184380 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.184381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.184505 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.185030 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:31.185039 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:31.186708 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:31.188426 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:31.188432 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:31.188434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:31.188659 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:31.188851 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:31.188851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:31.188852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:31.188960 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:31.189184 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:31.189189 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:31.189314 (+ 125us) client_negotiation.cc:769] Sending connection context
0923 22:11:31.189403 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:11:31.189521 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260923 22:11:31.190253 19453 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39565
I20260923 22:11:31.190348 19453 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:31.190554 19453 heartbeater.cc:511] Master 127.1.244.254:39565 requested a full tablet report, sending...
I20260923 22:11:31.190948 18994 ts_manager.cc:194] Registered new tserver with Master: bc993467a3b34ed68227d3db34239814 (127.1.244.195:42789)
I20260923 22:11:31.191354 18994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:34249
I20260923 22:11:31.197113 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:31.204192 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.198386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56866 (local address 127.1.244.254:39565)
0923 22:11:31.198448 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.198450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.198566 (+ 116us) server_negotiation.cc:407] Connection header received
0923 22:11:31.198603 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.198604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.198612 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.198660 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:31.199180 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.199693 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.200393 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.200569 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.200619 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.201747 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:31.201748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:31.201748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:31.201753 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:31.203102 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:31.203465 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:31.203465 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:31.203466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:31.203515 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:31.203747 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:31.203747 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:31.203748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:31.203884 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:31.203969 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.204048 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.204104 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:11:31.205982 18994 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:31.207538 18994 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:31.214596 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.210068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:34559 (local address 127.0.0.1:59700)
0923 22:11:31.210447 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.210456 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.210524 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.210791 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.210791 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.210793 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:31.210831 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.210832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.212192 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.212193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.213283 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.213286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.214270 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.214271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.214357 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.214374 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 22:11:31.214440 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:11:31.214498 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":39,"thread_start_us":109,"threads_started":1}
I20260923 22:11:31.214655 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.210568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.1.244.193:44905)
0923 22:11:31.210868 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.210873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.211042 (+ 169us) server_negotiation.cc:407] Connection header received
0923 22:11:31.211090 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.211092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.211104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.211192 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:31.211293 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.212062 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.213641 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.214337 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.214407 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.214459 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.214530 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.214571 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":75,"threads_started":1}
I20260923 22:11:31.214890 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.210341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.1.244.195:42789)
0923 22:11:31.210513 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.210517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.210528 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:11:31.210569 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.210571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.210587 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.210670 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:31.210918 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.212082 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.213652 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.214205 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.214280 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.214344 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.214601 (+ 257us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.214649 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260923 22:11:31.215027 19467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.210239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.1.244.194:34559)
0923 22:11:31.210424 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.210429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.210533 (+ 104us) server_negotiation.cc:407] Connection header received
0923 22:11:31.210614 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.210617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.210631 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.210708 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:31.210917 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.212078 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.213390 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.214103 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.214177 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.214233 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.214844 (+ 611us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.214881 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":113,"threads_started":1}
I20260923 22:11:31.214679 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.210063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42789 (local address 127.0.0.1:47242)
0923 22:11:31.210233 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.210274 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.210340 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.210676 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.210679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.210681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:31.210827 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.210829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.212207 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.212208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.213559 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.213562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.214326 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.214327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.214396 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.214400 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:31.214461 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:11:31.214509 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":93,"thread_start_us":65,"threads_started":1}
I20260923 22:11:31.214737 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.210073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44905 (local address 127.0.0.1:37286)
0923 22:11:31.210916 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.210927 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.210987 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.211189 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.211190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.211192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:31.211235 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.211237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.212167 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.212168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.213546 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.213549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.214406 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.214407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.214467 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.214470 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:31.214519 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:11:31.214558 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":107,"thread_start_us":94,"threads_started":1}
I20260923 22:11:31.215958 19387 tablet_service.cc:1467] Processing CreateTablet for tablet ed3a77e74a5f4c8a832d796a54b35b82 (DEFAULT_TABLE table=test-table [id=5c3fbe83c9d94ed5a67bb72f89c7b960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:31.216012 19253 tablet_service.cc:1467] Processing CreateTablet for tablet ed3a77e74a5f4c8a832d796a54b35b82 (DEFAULT_TABLE table=test-table [id=5c3fbe83c9d94ed5a67bb72f89c7b960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:31.216244 19387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed3a77e74a5f4c8a832d796a54b35b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:31.216154 19119 tablet_service.cc:1467] Processing CreateTablet for tablet ed3a77e74a5f4c8a832d796a54b35b82 (DEFAULT_TABLE table=test-table [id=5c3fbe83c9d94ed5a67bb72f89c7b960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:31.216360 19253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed3a77e74a5f4c8a832d796a54b35b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:31.216464 19119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed3a77e74a5f4c8a832d796a54b35b82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:31.218818 19470 tablet_bootstrap.cc:492] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: Bootstrap starting.
I20260923 22:11:31.219421 19470 tablet_bootstrap.cc:654] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:31.219655 19472 tablet_bootstrap.cc:492] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862: Bootstrap starting.
I20260923 22:11:31.220151 19470 log.cc:824] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:31.220345 19472 tablet_bootstrap.cc:654] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:31.220914 19472 log.cc:824] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:31.220989 19470 tablet_bootstrap.cc:492] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: No bootstrap required, opened a new log
I20260923 22:11:31.221046 19470 ts_tablet_manager.cc:1397] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:11:31.221257 19471 tablet_bootstrap.cc:492] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: Bootstrap starting.
I20260923 22:11:31.221676 19472 tablet_bootstrap.cc:492] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862: No bootstrap required, opened a new log
I20260923 22:11:31.221772 19472 ts_tablet_manager.cc:1397] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:11:31.221916 19471 tablet_bootstrap.cc:654] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:31.222467 19471 log.cc:824] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:31.223284 19471 tablet_bootstrap.cc:492] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: No bootstrap required, opened a new log
I20260923 22:11:31.223358 19471 ts_tablet_manager.cc:1397] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:31.223351 19470 raft_consensus.cc:348] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } }
I20260923 22:11:31.223534 19470 raft_consensus.cc:374] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:31.223606 19470 raft_consensus.cc:729] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc993467a3b34ed68227d3db34239814, State: Initialized, Role: FOLLOWER
I20260923 22:11:31.223501 19472 raft_consensus.cc:348] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } }
I20260923 22:11:31.223716 19472 raft_consensus.cc:374] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:31.223722 19470 consensus_queue.cc:260] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } }
I20260923 22:11:31.223764 19472 raft_consensus.cc:729] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffc8af4dcdb4648878dceb03d591862, State: Initialized, Role: FOLLOWER
I20260923 22:11:31.223874 19472 consensus_queue.cc:260] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } }
I20260923 22:11:31.223999 19470 ts_tablet_manager.cc:1428] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:11:31.224118 19453 heartbeater.cc:503] Master 127.1.244.254:39565 was elected leader, sending a full tablet report...
I20260923 22:11:31.224139 19472 ts_tablet_manager.cc:1428] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:11:31.224258 19185 heartbeater.cc:503] Master 127.1.244.254:39565 was elected leader, sending a full tablet report...
I20260923 22:11:31.225399 19471 raft_consensus.cc:348] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } }
I20260923 22:11:31.225530 19471 raft_consensus.cc:374] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:31.225553 19471 raft_consensus.cc:729] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18be22ee74a04cba91e3c04099c76467, State: Initialized, Role: FOLLOWER
I20260923 22:11:31.225684 19471 consensus_queue.cc:260] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } }
I20260923 22:11:31.225950 19471 ts_tablet_manager.cc:1428] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:31.225986 19319 heartbeater.cc:503] Master 127.1.244.254:39565 was elected leader, sending a full tablet report...
W20260923 22:11:31.276571 19320 tablet.cc:2369] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:11:31.365180 19186 tablet.cc:2369] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:31.380625 19477 raft_consensus.cc:482] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:31.380740 19477 raft_consensus.cc:504] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } }
I20260923 22:11:31.381153 19477 leader_election.cc:288] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc993467a3b34ed68227d3db34239814 (127.1.244.195:42789), 18be22ee74a04cba91e3c04099c76467 (127.1.244.194:34559)
I20260923 22:11:31.384274 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.381493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:57955 (local address 127.1.244.195:42789)
0923 22:11:31.381537 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.381539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.381695 (+ 156us) server_negotiation.cc:407] Connection header received
0923 22:11:31.381737 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.381738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.381748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.381829 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:31.382031 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.382492 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.383495 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.383910 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.383968 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.383975 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.384146 (+ 171us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.384190 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:11:31.384268 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.381393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42789 (local address 127.1.244.193:57955)
0923 22:11:31.381590 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.381600 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.381681 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.381843 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.381844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.381846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:31.381920 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.381921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.382607 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.382608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.383387 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.383389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.383995 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.383996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.384069 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.384074 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:11:31.384126 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:11:31.384165 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":4,"thread_start_us":80,"threads_started":1}
I20260923 22:11:31.384362 19467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.381471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:38799 (local address 127.1.244.194:34559)
0923 22:11:31.381523 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.381526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.381659 (+ 133us) server_negotiation.cc:407] Connection header received
0923 22:11:31.381693 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.381694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.381706 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.381757 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:31.381972 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.382468 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.383496 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.383967 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.384045 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.384055 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.384246 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.384290 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:11:31.384594 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.381393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:34559 (local address 127.1.244.193:38799)
0923 22:11:31.381541 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.381552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.381617 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.381803 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.381804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.381806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:31.381891 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.381894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.382560 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.382561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.383389 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.383391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.384077 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.384078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.384164 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.384168 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:31.384225 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:11:31.384263 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260923 22:11:31.384730 19407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3a77e74a5f4c8a832d796a54b35b82" candidate_uuid: "9ffc8af4dcdb4648878dceb03d591862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc993467a3b34ed68227d3db34239814" is_pre_election: true
I20260923 22:11:31.384897 19407 raft_consensus.cc:2454] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ffc8af4dcdb4648878dceb03d591862 in term 0.
I20260923 22:11:31.384938 19273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3a77e74a5f4c8a832d796a54b35b82" candidate_uuid: "9ffc8af4dcdb4648878dceb03d591862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18be22ee74a04cba91e3c04099c76467" is_pre_election: true
I20260923 22:11:31.385113 19273 raft_consensus.cc:2454] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ffc8af4dcdb4648878dceb03d591862 in term 0.
I20260923 22:11:31.385155 19072 leader_election.cc:302] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ffc8af4dcdb4648878dceb03d591862, bc993467a3b34ed68227d3db34239814; no voters:
I20260923 22:11:31.385272 19477 raft_consensus.cc:2781] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:31.385327 19477 raft_consensus.cc:482] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:31.385356 19477 raft_consensus.cc:3037] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:31.386073 19477 raft_consensus.cc:504] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } }
I20260923 22:11:31.386233 19477 leader_election.cc:288] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [CANDIDATE]: Term 1 election: Requested vote from peers bc993467a3b34ed68227d3db34239814 (127.1.244.195:42789), 18be22ee74a04cba91e3c04099c76467 (127.1.244.194:34559)
I20260923 22:11:31.386384 19407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3a77e74a5f4c8a832d796a54b35b82" candidate_uuid: "9ffc8af4dcdb4648878dceb03d591862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc993467a3b34ed68227d3db34239814"
I20260923 22:11:31.386384 19273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3a77e74a5f4c8a832d796a54b35b82" candidate_uuid: "9ffc8af4dcdb4648878dceb03d591862" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18be22ee74a04cba91e3c04099c76467"
I20260923 22:11:31.386461 19273 raft_consensus.cc:3037] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:31.386461 19407 raft_consensus.cc:3037] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:31.387336 19273 raft_consensus.cc:2454] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ffc8af4dcdb4648878dceb03d591862 in term 1.
I20260923 22:11:31.387336 19407 raft_consensus.cc:2454] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ffc8af4dcdb4648878dceb03d591862 in term 1.
I20260923 22:11:31.387511 19073 leader_election.cc:302] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18be22ee74a04cba91e3c04099c76467, 9ffc8af4dcdb4648878dceb03d591862; no voters:
I20260923 22:11:31.387626 19477 raft_consensus.cc:2781] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:31.387792 19477 raft_consensus.cc:686] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [term 1 LEADER]: Becoming Leader. State: Replica: 9ffc8af4dcdb4648878dceb03d591862, State: Running, Role: LEADER
I20260923 22:11:31.387897 19477 consensus_queue.cc:237] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } }
I20260923 22:11:31.388744 18994 catalog_manager.cc:5495] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ffc8af4dcdb4648878dceb03d591862 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "9ffc8af4dcdb4648878dceb03d591862" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ffc8af4dcdb4648878dceb03d591862" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44905 } health_report { overall_health: HEALTHY } } }
I20260923 22:11:31.395008 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.392894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.1.244.193:44905)
0923 22:11:31.392935 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.392938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.392951 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:11:31.392977 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.392977 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.392994 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.393053 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:31.393150 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.393612 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.394416 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.394608 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.394661 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.394772 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:31.394857 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:31.394910 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.394940 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.394964 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:11:31.396646 19273 raft_consensus.cc:1261] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [term 1 FOLLOWER]: Refusing update from remote peer 9ffc8af4dcdb4648878dceb03d591862: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:31.396660 19407 raft_consensus.cc:1261] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [term 1 FOLLOWER]: Refusing update from remote peer 9ffc8af4dcdb4648878dceb03d591862: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:31.397007 19481 consensus_queue.cc:1035] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18be22ee74a04cba91e3c04099c76467" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:31.397130 19477 consensus_queue.cc:1035] T ed3a77e74a5f4c8a832d796a54b35b82 P 9ffc8af4dcdb4648878dceb03d591862 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc993467a3b34ed68227d3db34239814" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:31.399312 19484 mvcc.cc:205] Tried to move back new op lower bound from 7332665308758347776 to 7332665308725579776. Current Snapshot: MvccSnapshot[applied={T|T < 7332665308758347776}]
I20260923 22:11:31.399516 19486 mvcc.cc:205] Tried to move back new op lower bound from 7332665308758347776 to 7332665308725579776. Current Snapshot: MvccSnapshot[applied={T|T < 7332665308758347776}]
I20260923 22:11:31.399821 19485 mvcc.cc:205] Tried to move back new op lower bound from 7332665308758347776 to 7332665308725579776. Current Snapshot: MvccSnapshot[applied={T|T < 7332665308758347776}]
I20260923 22:11:31.401615 18994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:31.401708 18994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:31.402446 18994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7cc058b3b44444dafbcb95870db879a: Sending DeleteTablet for 3 replicas of tablet ed3a77e74a5f4c8a832d796a54b35b82
I20260923 22:11:31.402769 19387 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3a77e74a5f4c8a832d796a54b35b82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:31 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:47242
I20260923 22:11:31.402814 19253 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3a77e74a5f4c8a832d796a54b35b82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:31 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:59700
I20260923 22:11:31.402997 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 19056
I20260923 22:11:31.403030 19494 tablet_replica.cc:330] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: stopping tablet replica
I20260923 22:11:31.403050 19119 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3a77e74a5f4c8a832d796a54b35b82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:31 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:37286
I20260923 22:11:31.403122 19494 raft_consensus.cc:2229] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:31.403218 19494 raft_consensus.cc:2258] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:31.404495 19494 ts_tablet_manager.cc:1905] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:31.406395 19494 ts_tablet_manager.cc:1918] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:31.406471 19494 log.cc:1196] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/ed3a77e74a5f4c8a832d796a54b35b82
I20260923 22:11:31.406638 19495 tablet_replica.cc:330] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: stopping tablet replica
I20260923 22:11:31.406737 19494 ts_tablet_manager.cc:1939] T ed3a77e74a5f4c8a832d796a54b35b82 P bc993467a3b34ed68227d3db34239814: Deleting consensus metadata
I20260923 22:11:31.406759 19495 raft_consensus.cc:2229] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:31.406860 19495 raft_consensus.cc:2258] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:31.407243 19495 ts_tablet_manager.cc:1905] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:31.407639 18978 catalog_manager.cc:4841] TS bc993467a3b34ed68227d3db34239814 (127.1.244.195:42789): tablet ed3a77e74a5f4c8a832d796a54b35b82 (table test-table [id=5c3fbe83c9d94ed5a67bb72f89c7b960]) successfully deleted
I20260923 22:11:31.408998 19495 ts_tablet_manager.cc:1918] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:31.409088 19495 log.cc:1196] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/ed3a77e74a5f4c8a832d796a54b35b82
I20260923 22:11:31.409320 19495 ts_tablet_manager.cc:1939] T ed3a77e74a5f4c8a832d796a54b35b82 P 18be22ee74a04cba91e3c04099c76467: Deleting consensus metadata
I20260923 22:11:31.409818 18979 catalog_manager.cc:4841] TS 18be22ee74a04cba91e3c04099c76467 (127.1.244.194:34559): tablet ed3a77e74a5f4c8a832d796a54b35b82 (table test-table [id=5c3fbe83c9d94ed5a67bb72f89c7b960]) successfully deleted
W20260923 22:11:31.410100 18981 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:44905 (error 108)
I20260923 22:11:31.410276 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 19190
I20260923 22:11:31.410990 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.410844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44905 (local address 127.0.0.1:37308)
0923 22:11:31.410912 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.410959 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:44905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260923 22:11:31.411118 18981 catalog_manager.cc:4584] TS 9ffc8af4dcdb4648878dceb03d591862 (127.1.244.193:44905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed3a77e74a5f4c8a832d796a54b35b82: Network error: Client connection negotiation failed: client connection to 127.1.244.193:44905: connect: Connection refused (error 111)
I20260923 22:11:31.415792 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 19324
I20260923 22:11:31.421231 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 18962
2026-09-23T22:11:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2048 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-23T22:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:31Z Disabled control of system clock
I20260923 22:11:31.449147 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38995
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:31.532580 19500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:31.532883 19500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:31.532908 19500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:31.534188 19500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:31.534250 19500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:31.534263 19500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:31.534322 19500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:31.534339 19500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:31.535763 19500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:31.536026 19500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:31.536335 19500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:31.541153 19506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:31.541320 19500 server_base.cc:1034] running on GCE node
W20260923 22:11:31.541143 19505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:31.541144 19508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:31.541712 19500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:31.541990 19500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:31.543138 19500 hybrid_clock.cc:648] HybridClock initialized: now 1790201491543123 us; error 29 us; skew 500 ppm
I20260923 22:11:31.544442 19500 webserver.cc:460] Webserver started at http://127.1.244.254:42761/ using document root <none> and password file <none>
I20260923 22:11:31.544689 19500 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:31.544735 19500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:31.544890 19500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:31.545981 19500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "5068e180cdbe41568e6af1e5b7e7876a"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "563c2ef3d25a6dbedc0bf364c4863b6a"
server_key_iv: "89d102109fcd2c6e47c720013835bcd1"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.546361 19500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "5068e180cdbe41568e6af1e5b7e7876a"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "563c2ef3d25a6dbedc0bf364c4863b6a"
server_key_iv: "89d102109fcd2c6e47c720013835bcd1"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.547688 19500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:31.548452 19514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.548666 19500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:31.548779 19500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "5068e180cdbe41568e6af1e5b7e7876a"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "563c2ef3d25a6dbedc0bf364c4863b6a"
server_key_iv: "89d102109fcd2c6e47c720013835bcd1"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.548873 19500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:31.563648 19500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:31.563947 19500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:31.564098 19500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:31.568040 19500 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:38995
I20260923 22:11:31.568073 19566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:38995 every 8 connection(s)
I20260923 22:11:31.568427 19500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:31.569425 19567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:31.571746 19567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a: Bootstrap starting.
I20260923 22:11:31.572324 19567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:31.572628 19567 log.cc:824] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:31.573386 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 19500
I20260923 22:11:31.573434 19567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a: No bootstrap required, opened a new log
I20260923 22:11:31.573500 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:31.573698 2003 external_mini_cluster.cc:1428] Setting key 7c1604d9f8704794f621d94eeeac1140
I20260923 22:11:31.575301 19567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5068e180cdbe41568e6af1e5b7e7876a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38995 } }
I20260923 22:11:31.575472 19567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:31.575522 19567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5068e180cdbe41568e6af1e5b7e7876a, State: Initialized, Role: FOLLOWER
I20260923 22:11:31.575675 19567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5068e180cdbe41568e6af1e5b7e7876a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38995 } }
I20260923 22:11:31.575778 19567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:31.575822 19567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:31.575891 19567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:31.576417 19567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5068e180cdbe41568e6af1e5b7e7876a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38995 } }
I20260923 22:11:31.576540 19567 leader_election.cc:302] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5068e180cdbe41568e6af1e5b7e7876a; no voters:
I20260923 22:11:31.576745 19567 leader_election.cc:288] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:31.576776 19572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:31.576879 19572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [term 1 LEADER]: Becoming Leader. State: Replica: 5068e180cdbe41568e6af1e5b7e7876a, State: Running, Role: LEADER
I20260923 22:11:31.576963 19572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5068e180cdbe41568e6af1e5b7e7876a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38995 } }
I20260923 22:11:31.577023 19567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:31.577301 19572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5068e180cdbe41568e6af1e5b7e7876a. Latest consensus state: current_term: 1 leader_uuid: "5068e180cdbe41568e6af1e5b7e7876a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5068e180cdbe41568e6af1e5b7e7876a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38995 } } }
I20260923 22:11:31.577316 19573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5068e180cdbe41568e6af1e5b7e7876a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5068e180cdbe41568e6af1e5b7e7876a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38995 } } }
I20260923 22:11:31.577378 19572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:31.577426 19573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:31.578605 19577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:31.579141 19577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:31.580842 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.574232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.1.244.254:38995)
0923 22:11:31.574464 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.574472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.574496 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:11:31.575290 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.575298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.575365 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.575558 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:31.575785 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.577387 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.578504 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.579119 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.579259 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.580070 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:31.580073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:31.580073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:31.580299 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:31.580354 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:31.580372 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:31.580453 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.580541 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.580625 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
I20260923 22:11:31.580998 19577 catalog_manager.cc:1347] Generated new cluster ID: f3939b3aa1774b719973f42295c950d4
I20260923 22:11:31.581051 19577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:31.597049 19577 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:31.597589 19577 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:31.601660 19577 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a: Generated new TSK 0
I20260923 22:11:31.601859 19577 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:31.604759 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38995
--builtin_ntp_servers=127.1.244.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:31.695336 19591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:31.695603 19591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:31.695672 19591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:31.696853 19591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:31.696892 19591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:31.696919 19591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:11:31.698279 19591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:31.698525 19591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:31.698845 19591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:31.703893 19597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:31.703903 19596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:31.703902 19599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:31.704351 19591 server_base.cc:1034] running on GCE node
I20260923 22:11:31.704563 19591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:31.704821 19591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:31.705997 19591 hybrid_clock.cc:648] HybridClock initialized: now 1790201491705984 us; error 32 us; skew 500 ppm
I20260923 22:11:31.707394 19591 webserver.cc:460] Webserver started at http://127.1.244.193:43253/ using document root <none> and password file <none>
I20260923 22:11:31.707698 19591 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:31.707772 19591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:31.707909 19591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:31.709044 19591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "d6e249fb5eef45be8ce8a1a17017d89d"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "0de46abd2ce84509a4fd9d3c7efa7d7c"
server_key_iv: "9c399c7f0ac22fa2934f49bcd2960d50"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.709434 19591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "d6e249fb5eef45be8ce8a1a17017d89d"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "0de46abd2ce84509a4fd9d3c7efa7d7c"
server_key_iv: "9c399c7f0ac22fa2934f49bcd2960d50"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.710804 19591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:31.711637 19605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.711822 19591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:31.711915 19591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "d6e249fb5eef45be8ce8a1a17017d89d"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "0de46abd2ce84509a4fd9d3c7efa7d7c"
server_key_iv: "9c399c7f0ac22fa2934f49bcd2960d50"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.712009 19591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:31.730600 19591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:31.730962 19591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:31.731129 19591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:31.731390 19591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:31.731769 19591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:31.731834 19591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.731877 19591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:31.731905 19591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.737957 19591 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:35099
I20260923 22:11:31.738030 19718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:35099 every 8 connection(s)
I20260923 22:11:31.738400 19591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:31.740427 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 19591
I20260923 22:11:31.740551 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:31.740775 2003 external_mini_cluster.cc:1428] Setting key 27ce409706c26f238ed7b71654d05756
I20260923 22:11:31.741827 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38995
--builtin_ntp_servers=127.1.244.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:11:31.744936 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.739234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:56341 (local address 127.1.244.254:38995)
0923 22:11:31.739316 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.739318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.739499 (+ 181us) server_negotiation.cc:407] Connection header received
0923 22:11:31.740062 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.740064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.740077 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.740142 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:31.741116 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.741692 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.743298 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.743653 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.743730 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.744604 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:31.744606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:31.744607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:31.744612 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:31.744626 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:31.744630 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:31.744678 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.744828 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.744858 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:11:31.745162 19721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.739147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38995 (local address 127.1.244.193:56341)
0923 22:11:31.739364 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.739406 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.739947 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.740205 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.740208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.740252 (+ 44us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:31.741042 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.741049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.741781 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.741782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.743157 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.743160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.743750 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.743751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.743868 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.744456 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:31.744456 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:31.744487 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:31.744510 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:31.744677 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:31.744683 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:31.744686 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:31.744826 (+ 140us) client_negotiation.cc:240] Negotiation successful
0923 22:11:31.744932 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260923 22:11:31.745668 19719 heartbeater.cc:347] Connected to a master server at 127.1.244.254:38995
I20260923 22:11:31.745766 19719 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:31.745975 19719 heartbeater.cc:511] Master 127.1.244.254:38995 requested a full tablet report, sending...
I20260923 22:11:31.746552 19531 ts_manager.cc:194] Registered new tserver with Master: d6e249fb5eef45be8ce8a1a17017d89d (127.1.244.193:35099)
I20260923 22:11:31.747562 19531 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.193:56341
W20260923 22:11:31.824303 19722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:31.824518 19722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:31.824540 19722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:31.825845 19722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:31.825901 19722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:31.825933 19722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:11:31.827420 19722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:31.827680 19722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:31.827976 19722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:31.833163 19728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:31.833294 19722 server_base.cc:1034] running on GCE node
W20260923 22:11:31.833150 19730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:31.833535 19727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:31.833775 19722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:31.834113 19722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:31.835268 19722 hybrid_clock.cc:648] HybridClock initialized: now 1790201491835229 us; error 52 us; skew 500 ppm
I20260923 22:11:31.836884 19722 webserver.cc:460] Webserver started at http://127.1.244.194:42689/ using document root <none> and password file <none>
I20260923 22:11:31.837173 19722 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:31.837221 19722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:31.837370 19722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:31.838485 19722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "7a0385a1e57e491395a4b18cb182d327"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "d14f6320041fa13bd7c4359c4d19751b"
server_key_iv: "59f9dfc8b044774bb96254c2df7003c9"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.838851 19722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "7a0385a1e57e491395a4b18cb182d327"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "d14f6320041fa13bd7c4359c4d19751b"
server_key_iv: "59f9dfc8b044774bb96254c2df7003c9"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.840282 19722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:11:31.841114 19736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.841297 19722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:31.841399 19722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "7a0385a1e57e491395a4b18cb182d327"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "d14f6320041fa13bd7c4359c4d19751b"
server_key_iv: "59f9dfc8b044774bb96254c2df7003c9"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.841495 19722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:31.861725 19722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:31.862084 19722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:31.862255 19722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:31.862524 19722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:31.862871 19722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:31.862927 19722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.862962 19722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:31.862980 19722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.868973 19722 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:46499
I20260923 22:11:31.869021 19849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:46499 every 8 connection(s)
I20260923 22:11:31.869405 19722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:31.874704 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.870205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:40685 (local address 127.1.244.254:38995)
0923 22:11:31.870282 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.870285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.870527 (+ 242us) server_negotiation.cc:407] Connection header received
0923 22:11:31.871048 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.871049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.871059 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.871116 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:31.871842 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.872328 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.873291 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.873486 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.873538 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.874444 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:31.874445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:31.874446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:31.874451 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:31.874466 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:31.874470 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:31.874530 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:31.874614 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 22:11:31.874639 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:11:31.874926 19852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.870180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38995 (local address 127.1.244.194:40685)
0923 22:11:31.870386 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.870437 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.870959 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.871178 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.871181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.871190 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:31.871767 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.871773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.872412 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.872413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.873211 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:31.873213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:31.873563 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:31.873563 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:31.873670 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:31.874282 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:31.874283 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:31.874314 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:31.874337 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:31.874540 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:31.874544 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:31.874546 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:31.874598 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:11:31.874682 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260923 22:11:31.875536 19850 heartbeater.cc:347] Connected to a master server at 127.1.244.254:38995
I20260923 22:11:31.875669 19850 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:31.875875 19850 heartbeater.cc:511] Master 127.1.244.254:38995 requested a full tablet report, sending...
I20260923 22:11:31.876293 19531 ts_manager.cc:194] Registered new tserver with Master: 7a0385a1e57e491395a4b18cb182d327 (127.1.244.194:46499)
I20260923 22:11:31.876732 19531 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.194:40685
I20260923 22:11:31.877723 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 19722
I20260923 22:11:31.877802 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:31.878001 2003 external_mini_cluster.cc:1428] Setting key fb65490a2e358b11fdee1fb667335f31
I20260923 22:11:31.878875 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38995
--builtin_ntp_servers=127.1.244.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:31.960826 19853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:31.961054 19853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:31.961078 19853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:31.962280 19853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:31.962328 19853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:31.962355 19853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:11:31.963842 19853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:31.964125 19853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:31.964462 19853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:31.969807 19861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:31.969830 19858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:31.970031 19853 server_base.cc:1034] running on GCE node
W20260923 22:11:31.969969 19859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:31.970314 19853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:31.970548 19853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:31.971715 19853 hybrid_clock.cc:648] HybridClock initialized: now 1790201491971693 us; error 37 us; skew 500 ppm
I20260923 22:11:31.973088 19853 webserver.cc:460] Webserver started at http://127.1.244.195:37445/ using document root <none> and password file <none>
I20260923 22:11:31.973349 19853 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:31.973397 19853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:31.973568 19853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:31.974687 19853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "71d6eff5b022483f9813d11ce3b6e4f8"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "23dcee8558d230c4040567a68e234803"
server_key_iv: "90c1f265fd6f62ee40446af03c143ed2"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.975066 19853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "71d6eff5b022483f9813d11ce3b6e4f8"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "23dcee8558d230c4040567a68e234803"
server_key_iv: "90c1f265fd6f62ee40446af03c143ed2"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.976619 19853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:11:31.977445 19867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.977650 19853 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:31.977798 19853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "71d6eff5b022483f9813d11ce3b6e4f8"
format_stamp: "Formatted at 2026-09-23 22:11:31 on dist-test-slave-b5jz"
server_key: "23dcee8558d230c4040567a68e234803"
server_key_iv: "90c1f265fd6f62ee40446af03c143ed2"
server_key_version: "encryptionkey@0"
I20260923 22:11:31.977883 19853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:31.989881 19853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:31.990242 19853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:31.990410 19853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:31.990650 19853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:31.991043 19853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:31.991106 19853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.991150 19853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:31.991171 19853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:31.997272 19853 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:40495
I20260923 22:11:31.997313 19980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:40495 every 8 connection(s)
I20260923 22:11:31.997676 19853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:32.003228 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.998481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:42987 (local address 127.1.244.254:38995)
0923 22:11:31.998562 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:31.998564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:31.998823 (+ 259us) server_negotiation.cc:407] Connection header received
0923 22:11:31.999359 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:31.999361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:31.999372 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:31.999438 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:32.000238 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.000775 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.001802 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.002016 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.002090 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.002994 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:32.002995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:32.002996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:32.003001 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:32.003015 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:32.003019 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:32.003061 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.003134 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.003161 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:11:32.003413 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.998488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38995 (local address 127.1.244.195:42987)
0923 22:11:31.998679 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.998723 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:31.999238 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:31.999502 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:31.999505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:31.999514 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:32.000125 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.000131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.000869 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.000870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.001694 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.001696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.002111 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.002111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.002232 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.002852 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:32.002852 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:32.002884 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:32.002908 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:32.003068 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:32.003071 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:32.003073 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:32.003122 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:11:32.003204 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260923 22:11:32.003722 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 19853
I20260923 22:11:32.003842 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:32.004061 2003 external_mini_cluster.cc:1428] Setting key 09f6c4af72f81aee2e2f4d8ca4096229
I20260923 22:11:32.004083 19981 heartbeater.cc:347] Connected to a master server at 127.1.244.254:38995
I20260923 22:11:32.004182 19981 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:32.004382 19981 heartbeater.cc:511] Master 127.1.244.254:38995 requested a full tablet report, sending...
I20260923 22:11:32.004738 19530 ts_manager.cc:194] Registered new tserver with Master: 71d6eff5b022483f9813d11ce3b6e4f8 (127.1.244.195:40495)
I20260923 22:11:32.005165 19530 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.195:42987
I20260923 22:11:32.016036 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:32.020514 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.017482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.1.244.254:38995)
0923 22:11:32.017563 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.017565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.017630 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:11:32.017684 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.017684 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.017696 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.017742 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:32.017873 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.018355 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.019104 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.019289 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.019351 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.020224 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:32.020225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:32.020225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:32.020230 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:32.020244 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:32.020248 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:32.020305 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.020357 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.020383 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 22:11:32.022460 19530 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:32.024137 19530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:32.031270 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.026980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33358 (local address 127.1.244.194:46499)
0923 22:11:32.027722 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.027726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.027734 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:11:32.027765 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.027767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.027779 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.027870 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:32.027976 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.028670 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.030361 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.030843 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.030946 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.031034 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.031125 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.031169 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":80,"threads_started":1}
I20260923 22:11:32.031273 19990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.026616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:35099 (local address 127.0.0.1:54066)
0923 22:11:32.027029 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:32.027075 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:32.027147 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:32.027453 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:32.027453 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:32.027455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:32.027517 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.027520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.028920 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.028921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.030123 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.030126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.031007 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.031008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.031079 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.031085 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:11:32.031118 (+ 33us) client_negotiation.cc:240] Negotiation successful
0923 22:11:32.031189 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":73,"spinlock_wait_cycles":768,"thread_start_us":110,"threads_started":1}
I20260923 22:11:32.031405 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.026824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.1.244.193:35099)
0923 22:11:32.027007 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.027013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.027260 (+ 247us) server_negotiation.cc:407] Connection header received
0923 22:11:32.027306 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.027308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.027323 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.027388 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:32.027664 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.028824 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.030215 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.030898 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.031165 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.031260 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.031286 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.031328 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260923 22:11:32.031697 19989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.026872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:46499 (local address 127.0.0.1:33358)
0923 22:11:32.027056 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:32.027075 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:32.027147 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:32.027867 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:32.027868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:32.027869 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:32.027919 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.027920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.028776 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.028777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.030271 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.030273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.030950 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.030952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.031029 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.031043 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:11:32.031114 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:11:32.031624 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":73,"thread_start_us":85,"threads_started":1}
I20260923 22:11:32.031935 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.026723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42246 (local address 127.1.244.195:40495)
0923 22:11:32.027188 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.027193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.027204 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:11:32.027248 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.027250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.027266 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.027345 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:32.027654 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.028830 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.030032 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.030803 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.030878 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.030978 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.031315 (+ 337us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.031359 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":137,"threads_started":1}
I20260923 22:11:32.031278 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.026617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40495 (local address 127.0.0.1:42246)
0923 22:11:32.027029 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:32.027075 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:32.027148 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:32.027429 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:32.027433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:32.027435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:32.027517 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.027519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.028930 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.028931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.029930 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.029933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.031009 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.031009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.031079 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.031086 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:11:32.031135 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:11:32.031181 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":15,"thread_start_us":87,"threads_started":1}
I20260923 22:11:32.032378 19653 tablet_service.cc:1467] Processing CreateTablet for tablet 0968925b08644ce5807783ab73882719 (DEFAULT_TABLE table=test-table [id=2e82bea4b7204b4d980dbb91d8cf4056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:32.032558 19784 tablet_service.cc:1467] Processing CreateTablet for tablet 0968925b08644ce5807783ab73882719 (DEFAULT_TABLE table=test-table [id=2e82bea4b7204b4d980dbb91d8cf4056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:32.032787 19653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0968925b08644ce5807783ab73882719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:32.032866 19784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0968925b08644ce5807783ab73882719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:32.036329 19995 tablet_bootstrap.cc:492] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d: Bootstrap starting.
I20260923 22:11:32.036715 19996 tablet_bootstrap.cc:492] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: Bootstrap starting.
I20260923 22:11:32.036716 19915 tablet_service.cc:1467] Processing CreateTablet for tablet 0968925b08644ce5807783ab73882719 (DEFAULT_TABLE table=test-table [id=2e82bea4b7204b4d980dbb91d8cf4056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:32.037057 19915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0968925b08644ce5807783ab73882719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:32.037065 19995 tablet_bootstrap.cc:654] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:32.037398 19996 tablet_bootstrap.cc:654] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:32.037459 19995 log.cc:824] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:32.037716 19996 log.cc:824] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:32.038298 19995 tablet_bootstrap.cc:492] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d: No bootstrap required, opened a new log
I20260923 22:11:32.038393 19995 ts_tablet_manager.cc:1397] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:11:32.038537 19996 tablet_bootstrap.cc:492] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: No bootstrap required, opened a new log
I20260923 22:11:32.038609 19996 ts_tablet_manager.cc:1397] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:32.040414 19995 raft_consensus.cc:348] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } }
I20260923 22:11:32.040589 19995 raft_consensus.cc:374] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:32.040638 19995 raft_consensus.cc:729] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e249fb5eef45be8ce8a1a17017d89d, State: Initialized, Role: FOLLOWER
I20260923 22:11:32.040763 19995 consensus_queue.cc:260] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } }
I20260923 22:11:32.040855 19999 tablet_bootstrap.cc:492] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: Bootstrap starting.
I20260923 22:11:32.041034 19995 ts_tablet_manager.cc:1428] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:11:32.041185 19719 heartbeater.cc:503] Master 127.1.244.254:38995 was elected leader, sending a full tablet report...
I20260923 22:11:32.041275 19996 raft_consensus.cc:348] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } }
I20260923 22:11:32.041442 19996 raft_consensus.cc:374] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:32.041494 19996 raft_consensus.cc:729] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0385a1e57e491395a4b18cb182d327, State: Initialized, Role: FOLLOWER
I20260923 22:11:32.041518 19999 tablet_bootstrap.cc:654] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:32.041651 19996 consensus_queue.cc:260] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } }
I20260923 22:11:32.041843 19999 log.cc:824] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:32.041939 19996 ts_tablet_manager.cc:1428] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260923 22:11:32.042047 19850 heartbeater.cc:503] Master 127.1.244.254:38995 was elected leader, sending a full tablet report...
I20260923 22:11:32.042609 19999 tablet_bootstrap.cc:492] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: No bootstrap required, opened a new log
I20260923 22:11:32.042676 19999 ts_tablet_manager.cc:1397] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:32.044927 19999 raft_consensus.cc:348] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } }
I20260923 22:11:32.045085 19999 raft_consensus.cc:374] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:32.045133 19999 raft_consensus.cc:729] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d6eff5b022483f9813d11ce3b6e4f8, State: Initialized, Role: FOLLOWER
I20260923 22:11:32.045261 19999 consensus_queue.cc:260] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } }
I20260923 22:11:32.045540 19999 ts_tablet_manager.cc:1428] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:32.045620 19981 heartbeater.cc:503] Master 127.1.244.254:38995 was elected leader, sending a full tablet report...
W20260923 22:11:32.119966 19851 tablet.cc:2369] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:11:32.239153 19720 tablet.cc:2369] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:32.242436 20001 raft_consensus.cc:482] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:32.242570 20001 raft_consensus.cc:504] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } }
I20260923 22:11:32.242933 20001 leader_election.cc:288] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71d6eff5b022483f9813d11ce3b6e4f8 (127.1.244.195:40495), d6e249fb5eef45be8ce8a1a17017d89d (127.1.244.193:35099)
I20260923 22:11:32.246131 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.243197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:40383 (local address 127.1.244.195:40495)
0923 22:11:32.243284 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.243286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.243318 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:11:32.243355 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.243356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.243366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.243423 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:32.243623 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.244157 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.245190 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.245729 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.245787 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.245793 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.246015 (+ 222us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.246057 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 22:11:32.246132 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.243208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:34947 (local address 127.1.244.193:35099)
0923 22:11:32.243285 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.243287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.243423 (+ 136us) server_negotiation.cc:407] Connection header received
0923 22:11:32.243452 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.243454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.243466 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.243531 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:32.243747 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.244327 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.245397 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.245786 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.245869 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.245877 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.246028 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.246063 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:11:32.246143 19852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.243142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40495 (local address 127.1.244.194:40383)
0923 22:11:32.243203 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:32.243215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:32.243263 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:32.243443 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:32.243444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:32.243446 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:32.243529 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.243530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.244252 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.244253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.245111 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.245113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.245842 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.245844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.245920 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.245923 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:32.246001 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:11:32.246047 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:11:32.246157 20004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.243155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:35099 (local address 127.1.244.194:34947)
0923 22:11:32.243323 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:32.243334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:32.243397 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:32.243588 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:32.243589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:32.243590 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:32.243667 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.243669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.244412 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.244413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.245277 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.245280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.245874 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.245875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.245931 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.245934 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:32.246020 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:11:32.246057 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":11,"thread_start_us":99,"threads_started":1}
I20260923 22:11:32.246531 19673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968925b08644ce5807783ab73882719" candidate_uuid: "7a0385a1e57e491395a4b18cb182d327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" is_pre_election: true
I20260923 22:11:32.246703 19673 raft_consensus.cc:2454] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0385a1e57e491395a4b18cb182d327 in term 0.
I20260923 22:11:32.246755 19935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968925b08644ce5807783ab73882719" candidate_uuid: "7a0385a1e57e491395a4b18cb182d327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" is_pre_election: true
I20260923 22:11:32.246924 19935 raft_consensus.cc:2454] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0385a1e57e491395a4b18cb182d327 in term 0.
I20260923 22:11:32.246932 19739 leader_election.cc:302] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0385a1e57e491395a4b18cb182d327, d6e249fb5eef45be8ce8a1a17017d89d; no voters:
I20260923 22:11:32.247061 20001 raft_consensus.cc:2781] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:32.247134 20001 raft_consensus.cc:482] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:32.247164 20001 raft_consensus.cc:3037] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:32.247970 20001 raft_consensus.cc:504] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } }
I20260923 22:11:32.248122 20001 leader_election.cc:288] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [CANDIDATE]: Term 1 election: Requested vote from peers 71d6eff5b022483f9813d11ce3b6e4f8 (127.1.244.195:40495), d6e249fb5eef45be8ce8a1a17017d89d (127.1.244.193:35099)
W20260923 22:11:32.248242 19982 tablet.cc:2369] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:32.248320 19673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968925b08644ce5807783ab73882719" candidate_uuid: "7a0385a1e57e491395a4b18cb182d327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6e249fb5eef45be8ce8a1a17017d89d"
I20260923 22:11:32.248431 19673 raft_consensus.cc:3037] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:32.248667 19935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0968925b08644ce5807783ab73882719" candidate_uuid: "7a0385a1e57e491395a4b18cb182d327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d6eff5b022483f9813d11ce3b6e4f8"
I20260923 22:11:32.248735 19935 raft_consensus.cc:3037] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:32.249261 19673 raft_consensus.cc:2454] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0385a1e57e491395a4b18cb182d327 in term 1.
I20260923 22:11:32.249444 19739 leader_election.cc:302] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0385a1e57e491395a4b18cb182d327, d6e249fb5eef45be8ce8a1a17017d89d; no voters:
I20260923 22:11:32.249459 19935 raft_consensus.cc:2454] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0385a1e57e491395a4b18cb182d327 in term 1.
I20260923 22:11:32.249576 20001 raft_consensus.cc:2781] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:32.249748 20001 raft_consensus.cc:686] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0385a1e57e491395a4b18cb182d327, State: Running, Role: LEADER
I20260923 22:11:32.249853 20001 consensus_queue.cc:237] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } }
I20260923 22:11:32.250667 19530 catalog_manager.cc:5495] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0385a1e57e491395a4b18cb182d327 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "7a0385a1e57e491395a4b18cb182d327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0385a1e57e491395a4b18cb182d327" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 46499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 } health_report { overall_health: UNKNOWN } } }
I20260923 22:11:32.263180 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.260993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.1.244.194:46499)
0923 22:11:32.261056 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.261058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.261067 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:11:32.261101 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.261102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.261119 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.261183 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:32.261290 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.261751 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.262442 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.262630 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.262690 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.262836 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:32.262934 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:32.262997 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.263030 (+ 33us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.263049 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:11:32.265048 19935 raft_consensus.cc:1261] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [term 1 FOLLOWER]: Refusing update from remote peer 7a0385a1e57e491395a4b18cb182d327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:32.265048 19673 raft_consensus.cc:1261] T 0968925b08644ce5807783ab73882719 P d6e249fb5eef45be8ce8a1a17017d89d [term 1 FOLLOWER]: Refusing update from remote peer 7a0385a1e57e491395a4b18cb182d327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:32.265424 20001 consensus_queue.cc:1035] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71d6eff5b022483f9813d11ce3b6e4f8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:32.265602 20005 consensus_queue.cc:1035] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6e249fb5eef45be8ce8a1a17017d89d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:32.268142 20008 mvcc.cc:205] Tried to move back new op lower bound from 7332665312314802176 to 7332665312255799296. Current Snapshot: MvccSnapshot[applied={T|T < 7332665312314802176}]
I20260923 22:11:32.269045 20009 mvcc.cc:205] Tried to move back new op lower bound from 7332665312314802176 to 7332665312255799296. Current Snapshot: MvccSnapshot[applied={T|T < 7332665312314802176 or (T in {7332665312314802176})}]
I20260923 22:11:32.270265 19531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:32.270375 19531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:32.271166 19531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5068e180cdbe41568e6af1e5b7e7876a: Sending DeleteTablet for 3 replicas of tablet 0968925b08644ce5807783ab73882719
I20260923 22:11:32.271521 19784 tablet_service.cc:1514] Processing DeleteTablet for tablet 0968925b08644ce5807783ab73882719 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:32 UTC) from {username='slave'} at 127.0.0.1:33358
I20260923 22:11:32.271574 19653 tablet_service.cc:1514] Processing DeleteTablet for tablet 0968925b08644ce5807783ab73882719 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:32 UTC) from {username='slave'} at 127.0.0.1:54066
I20260923 22:11:32.271785 19915 tablet_service.cc:1514] Processing DeleteTablet for tablet 0968925b08644ce5807783ab73882719 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:32 UTC) from {username='slave'} at 127.0.0.1:42246
I20260923 22:11:32.271865 20018 tablet_replica.cc:330] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: stopping tablet replica
I20260923 22:11:32.271955 20018 raft_consensus.cc:2229] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:11:32.272022 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 19591
I20260923 22:11:32.272181 20018 raft_consensus.cc:2258] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:32.272672 20018 ts_tablet_manager.cc:1905] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:32.274523 20018 ts_tablet_manager.cc:1918] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:32.274587 20018 log.cc:1196] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/0968925b08644ce5807783ab73882719
I20260923 22:11:32.274852 20018 ts_tablet_manager.cc:1939] T 0968925b08644ce5807783ab73882719 P 7a0385a1e57e491395a4b18cb182d327: Deleting consensus metadata
I20260923 22:11:32.275270 19515 catalog_manager.cc:4841] TS 7a0385a1e57e491395a4b18cb182d327 (127.1.244.194:46499): tablet 0968925b08644ce5807783ab73882719 (table test-table [id=2e82bea4b7204b4d980dbb91d8cf4056]) successfully deleted
W20260923 22:11:32.278613 19517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:35099 (error 108)
I20260923 22:11:32.278869 20020 tablet_replica.cc:330] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: stopping tablet replica
I20260923 22:11:32.279042 20020 raft_consensus.cc:2229] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:32.279178 20020 raft_consensus.cc:2258] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:32.279184 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.279046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:35099 (local address 127.0.0.1:54074)
0923 22:11:32.279094 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:32.279147 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:35099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 22:11:32.279320 19517 catalog_manager.cc:4584] TS d6e249fb5eef45be8ce8a1a17017d89d (127.1.244.193:35099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0968925b08644ce5807783ab73882719: Network error: Client connection negotiation failed: client connection to 127.1.244.193:35099: connect: Connection refused (error 111)
I20260923 22:11:32.279384 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 19722
I20260923 22:11:32.280673 20020 ts_tablet_manager.cc:1905] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:32.282269 20020 ts_tablet_manager.cc:1918] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:32.282330 20020 log.cc:1196] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/0968925b08644ce5807783ab73882719
I20260923 22:11:32.282569 20020 ts_tablet_manager.cc:1939] T 0968925b08644ce5807783ab73882719 P 71d6eff5b022483f9813d11ce3b6e4f8: Deleting consensus metadata
I20260923 22:11:32.283039 19515 catalog_manager.cc:4841] TS 71d6eff5b022483f9813d11ce3b6e4f8 (127.1.244.195:40495): tablet 0968925b08644ce5807783ab73882719 (table test-table [id=2e82bea4b7204b4d980dbb91d8cf4056]) successfully deleted
I20260923 22:11:32.284911 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 19853
I20260923 22:11:32.289837 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 19500
2026-09-23T22:11:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (867 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:32 dist-test-slave-b5jz krb5kdc[20024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:32 dist-test-slave-b5jz krb5kdc[20024](info): set up 2 sockets
Sep 23 22:11:32 dist-test-slave-b5jz krb5kdc[20024](info): commencing operation
krb5kdc: starting...
W20260923 22:11:33.323293 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:33 dist-test-slave-b5jz krb5kdc[20024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:11:33.442436 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41733
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:41733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:33.525547 20037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:33.525761 20037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:33.525790 20037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:33.527067 20037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:33.527110 20037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:33.527124 20037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:33.527137 20037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:33.527149 20037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:33.528607 20037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:41733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:33.528820 20037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:33.528999 20037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:33.533759 20043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.533802 20042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.533787 20045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.534138 20037 server_base.cc:1034] running on GCE node
I20260923 22:11:33.534411 20037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:33.534723 20037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:33.535912 20037 hybrid_clock.cc:648] HybridClock initialized: now 1790201493535890 us; error 38 us; skew 500 ppm
Sep 23 22:11:33 dist-test-slave-b5jz krb5kdc[20024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:33.538380 20037 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:11:33.538873 20037 webserver.cc:460] Webserver started at http://127.1.244.254:44175/ using document root <none> and password file <none>
I20260923 22:11:33.539136 20037 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:33.539208 20037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:33.539359 20037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:33.540512 20037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "1346733eb5cb4d02b2c7f912f932ff03"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-b5jz"
server_key: "3ef83cf18bb549014d484edff3dd1d5d"
server_key_iv: "151b164efd4e58a340d8d830855e995a"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.540889 20037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "1346733eb5cb4d02b2c7f912f932ff03"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-b5jz"
server_key: "3ef83cf18bb549014d484edff3dd1d5d"
server_key_iv: "151b164efd4e58a340d8d830855e995a"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.542270 20037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:33.543023 20052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.543202 20037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:33.543306 20037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "1346733eb5cb4d02b2c7f912f932ff03"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-b5jz"
server_key: "3ef83cf18bb549014d484edff3dd1d5d"
server_key_iv: "151b164efd4e58a340d8d830855e995a"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.543395 20037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:33.558590 20037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:33.560820 20037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:33.561010 20037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:33.565564 20037 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:41733
I20260923 22:11:33.565596 20104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:41733 every 8 connection(s)
I20260923 22:11:33.565990 20037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:33.566869 20105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:33.567587 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 20037
I20260923 22:11:33.567703 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:33.567901 2003 external_mini_cluster.cc:1428] Setting key 14d216dba19f632b676264f5d9f73777
I20260923 22:11:33.569761 20105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03: Bootstrap starting.
I20260923 22:11:33.570359 20105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:33.570672 20105 log.cc:824] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:33.571357 20105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03: No bootstrap required, opened a new log
I20260923 22:11:33.573418 20105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1346733eb5cb4d02b2c7f912f932ff03" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41733 } }
I20260923 22:11:33.573596 20105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:33.573648 20105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1346733eb5cb4d02b2c7f912f932ff03, State: Initialized, Role: FOLLOWER
I20260923 22:11:33.573805 20105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1346733eb5cb4d02b2c7f912f932ff03" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41733 } }
I20260923 22:11:33.573905 20105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:33.573951 20105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:33.574000 20105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:33.574714 20105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1346733eb5cb4d02b2c7f912f932ff03" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41733 } }
I20260923 22:11:33.574831 20105 leader_election.cc:302] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1346733eb5cb4d02b2c7f912f932ff03; no voters:
Sep 23 22:11:33 dist-test-slave-b5jz krb5kdc[20024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:33.575008 20105 leader_election.cc:288] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:33.575063 20110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:33.575237 20110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [term 1 LEADER]: Becoming Leader. State: Replica: 1346733eb5cb4d02b2c7f912f932ff03, State: Running, Role: LEADER
I20260923 22:11:33.575302 20105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:33.575332 20110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1346733eb5cb4d02b2c7f912f932ff03" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41733 } }
I20260923 22:11:33.575711 20110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1346733eb5cb4d02b2c7f912f932ff03. Latest consensus state: current_term: 1 leader_uuid: "1346733eb5cb4d02b2c7f912f932ff03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1346733eb5cb4d02b2c7f912f932ff03" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41733 } } }
I20260923 22:11:33.575815 20110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:33.575726 20112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1346733eb5cb4d02b2c7f912f932ff03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1346733eb5cb4d02b2c7f912f932ff03" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41733 } } }
I20260923 22:11:33.576013 20112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:33.576061 20117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:33.577039 20117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:33.578583 20117 catalog_manager.cc:1347] Generated new cluster ID: b92b211dab3b4b7e87bbdffbf7cce2c2
I20260923 22:11:33.578634 20117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:33.579695 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.568484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55410 (local address 127.1.244.254:41733)
0923 22:11:33.568675 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.568689 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.568717 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:11:33.569114 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.569120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.569167 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.569327 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.570040 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.571177 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.572053 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.572497 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.572638 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.575500 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.575504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.575505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:33.575522 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.577950 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.578379 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.578383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.578383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.578495 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.578729 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.578730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.578730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.579037 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.579186 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.579317 (+ 131us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.579462 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260923 22:11:33.594128 20117 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:33.594738 20117 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:33.600618 20117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03: Generated new TSK 0
I20260923 22:11:33.600914 20117 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:11:33.634100 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41733
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:33.716188 20131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:33.716444 20131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:33.716496 20131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:33.717679 20131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:33.717756 20131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:33.717803 20131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:11:33.719307 20131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:33.719612 20131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:33.719849 20131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:33.725040 20136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.725033 20139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.725068 20131 server_base.cc:1034] running on GCE node
W20260923 22:11:33.725028 20137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.725469 20131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:33.725765 20131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:33.726925 20131 hybrid_clock.cc:648] HybridClock initialized: now 1790201493726904 us; error 38 us; skew 500 ppm
Sep 23 22:11:33 dist-test-slave-b5jz krb5kdc[20024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:33.729552 20131 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:11:33.729945 20131 webserver.cc:460] Webserver started at http://127.1.244.193:34249/ using document root <none> and password file <none>
I20260923 22:11:33.730196 20131 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:33.730265 20131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:33.730392 20131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:33.731413 20131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "fc2cdb4a591a4f5ca518f35da92ead90"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-b5jz"
server_key: "ad1e75f94235a0a431c713957f00b71e"
server_key_iv: "6bd85a44fb090b44f07311a01991a284"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.731827 20131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "fc2cdb4a591a4f5ca518f35da92ead90"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-b5jz"
server_key: "ad1e75f94235a0a431c713957f00b71e"
server_key_iv: "6bd85a44fb090b44f07311a01991a284"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.733101 20131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:33.733831 20146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.734066 20131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:33.734154 20131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "fc2cdb4a591a4f5ca518f35da92ead90"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-b5jz"
server_key: "ad1e75f94235a0a431c713957f00b71e"
server_key_iv: "6bd85a44fb090b44f07311a01991a284"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.734228 20131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:33.754359 20131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:33.756503 20131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:33.756668 20131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:33.756934 20131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:33.757267 20131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:33.757301 20131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.757324 20131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:33.757373 20131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.763033 20131 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:41047
I20260923 22:11:33.763175 20259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:41047 every 8 connection(s)
I20260923 22:11:33.763481 20131 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:33.769611 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 20131
I20260923 22:11:33.769713 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:33.769942 2003 external_mini_cluster.cc:1428] Setting key 87345fd3681f8a8e1bed39bf552a9d34
Sep 23 22:11:33 dist-test-slave-b5jz krb5kdc[20024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:33.773483 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.764308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:35503 (local address 127.1.244.254:41733)
0923 22:11:33.764392 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.764394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.764592 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:11:33.765162 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.765164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.765176 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.765235 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.766247 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.766776 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.767920 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.768145 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.768230 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.770512 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.770514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.770514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:33.770522 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.772005 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.772432 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.772433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.772434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.772504 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.772751 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.772752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.772752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.772903 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.773019 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.773329 (+ 310us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.773410 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:11:33.773763 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.764261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41733 (local address 127.1.244.193:35503)
0923 22:11:33.764446 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.764502 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.765047 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.765313 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.765318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.765922 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:33.766133 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.766137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.766883 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.766884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.767823 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.767826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.768238 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.768239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.768379 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.768891 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:33.768900 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:33.770402 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:33.772106 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:33.772114 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:33.772116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:33.772355 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:33.772574 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:33.772574 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:33.772574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:33.772682 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:33.773005 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:33.773010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:33.773184 (+ 174us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.773296 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.773480 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260923 22:11:33.774362 20260 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41733
I20260923 22:11:33.774497 20260 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:33.774695 20260 heartbeater.cc:511] Master 127.1.244.254:41733 requested a full tablet report, sending...
I20260923 22:11:33.775314 20069 ts_manager.cc:194] Registered new tserver with Master: fc2cdb4a591a4f5ca518f35da92ead90 (127.1.244.193:41047)
I20260923 22:11:33.776087 20069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:35503
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:11:33.794545 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41733
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:33.880074 20265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:33.880286 20265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:33.880311 20265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:33.881500 20265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:33.881542 20265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:33.881597 20265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:11:33.883051 20265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:33.883306 20265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:33.883595 20265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:33.888623 20270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.888607 20273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.888633 20271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.888962 20265 server_base.cc:1034] running on GCE node
I20260923 22:11:33.889183 20265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:33.889406 20265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:33.890578 20265 hybrid_clock.cc:648] HybridClock initialized: now 1790201493890565 us; error 27 us; skew 500 ppm
Sep 23 22:11:33 dist-test-slave-b5jz krb5kdc[20024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:33.893324 20265 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:11:33.893728 20265 webserver.cc:460] Webserver started at http://127.1.244.194:34399/ using document root <none> and password file <none>
I20260923 22:11:33.893975 20265 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:33.894049 20265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:33.894202 20265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:33.895283 20265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "37a5b0afea7d459a84f5b4ec8e817f04"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-b5jz"
server_key: "06a465b4405eae571fbbf21225b7b5aa"
server_key_iv: "3aaf56dc155d001260ece360d2bd1f57"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.895707 20265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "37a5b0afea7d459a84f5b4ec8e817f04"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-b5jz"
server_key: "06a465b4405eae571fbbf21225b7b5aa"
server_key_iv: "3aaf56dc155d001260ece360d2bd1f57"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.897182 20265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:33.897971 20280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.898164 20265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:33.898268 20265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "37a5b0afea7d459a84f5b4ec8e817f04"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-b5jz"
server_key: "06a465b4405eae571fbbf21225b7b5aa"
server_key_iv: "3aaf56dc155d001260ece360d2bd1f57"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.898351 20265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:33.925275 20265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:33.927479 20265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:33.927690 20265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:33.927953 20265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:33.928289 20265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:33.928347 20265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.928390 20265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:33.928412 20265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.934443 20265 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:42987
I20260923 22:11:33.934508 20393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:42987 every 8 connection(s)
I20260923 22:11:33.934870 20265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:33.939813 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 20265
I20260923 22:11:33.939949 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:33.940165 2003 external_mini_cluster.cc:1428] Setting key 2c8e4f9e6a74847d3591d8380f9d9f80
Sep 23 22:11:33 dist-test-slave-b5jz krb5kdc[20024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:33.946031 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.935713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:55603 (local address 127.1.244.254:41733)
0923 22:11:33.935830 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.935833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.936086 (+ 253us) server_negotiation.cc:407] Connection header received
0923 22:11:33.936753 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.936755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.936769 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.936845 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.937981 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.938680 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.939766 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.940035 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.940111 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.942581 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.942583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.942583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:33.942590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.944453 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.944918 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.944919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.944920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.944990 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.945233 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.945234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.945234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.945396 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.945472 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.945857 (+ 385us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.945927 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260923 22:11:33.946314 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.935708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41733 (local address 127.1.244.194:55603)
0923 22:11:33.935920 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.935979 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.936531 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.936991 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.936995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.937668 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:33.937872 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.937879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.938768 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.938769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.939665 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.939668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.940119 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.940120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.940274 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.940862 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:33.940873 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:33.942477 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:33.944563 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:33.944572 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:33.944575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:33.944832 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:33.945063 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:33.945063 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:33.945064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:33.945164 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:33.945506 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:33.945513 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:33.945697 (+ 184us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.945830 (+ 133us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.946024 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260923 22:11:33.947010 20394 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41733
I20260923 22:11:33.947132 20394 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:33.947335 20394 heartbeater.cc:511] Master 127.1.244.254:41733 requested a full tablet report, sending...
I20260923 22:11:33.947854 20069 ts_manager.cc:194] Registered new tserver with Master: 37a5b0afea7d459a84f5b4ec8e817f04 (127.1.244.194:42987)
I20260923 22:11:33.948243 20069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:55603
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:11:33.967464 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41733
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:34.071707 20399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:34.071981 20399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:34.072026 20399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:34.073338 20399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:34.073431 20399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:34.073478 20399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:11:34.075189 20399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:34.075484 20399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:34.075762 20399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:34.081430 20404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:34.081457 20399 server_base.cc:1034] running on GCE node
W20260923 22:11:34.081463 20405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:34.081491 20407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:34.081946 20399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:34.082237 20399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:34.083420 20399 hybrid_clock.cc:648] HybridClock initialized: now 1790201494083418 us; error 42 us; skew 500 ppm
Sep 23 22:11:34 dist-test-slave-b5jz krb5kdc[20024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201494, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:34.086073 20399 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:11:34.086575 20399 webserver.cc:460] Webserver started at http://127.1.244.195:44657/ using document root <none> and password file <none>
I20260923 22:11:34.086809 20399 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:34.086855 20399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:34.087014 20399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:34.088165 20399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "a75d87bfceaa42718657368a62685709"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "b4807b2432c63320d1c04eaccc410883"
server_key_iv: "054c748a5280441ddfc3d46a52f0a866"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.088527 20399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "a75d87bfceaa42718657368a62685709"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "b4807b2432c63320d1c04eaccc410883"
server_key_iv: "054c748a5280441ddfc3d46a52f0a866"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.089871 20399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:34.090708 20414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.090939 20399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:34.091041 20399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "a75d87bfceaa42718657368a62685709"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "b4807b2432c63320d1c04eaccc410883"
server_key_iv: "054c748a5280441ddfc3d46a52f0a866"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.091122 20399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:34.100939 20399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:34.103066 20399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:34.103228 20399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:34.103458 20399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:34.103845 20399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:34.103911 20399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.103955 20399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:34.103981 20399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.110006 20399 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:44001
I20260923 22:11:34.110049 20527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:44001 every 8 connection(s)
I20260923 22:11:34.110369 20399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:34.112748 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 20399
I20260923 22:11:34.112867 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:34.113080 2003 external_mini_cluster.cc:1428] Setting key 9eaa510e18ec190afbea6486e66b22a9
Sep 23 22:11:34 dist-test-slave-b5jz krb5kdc[20024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201494, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:34.119940 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.111082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:46983 (local address 127.1.244.254:41733)
0923 22:11:34.111128 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.111130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.111331 (+ 201us) server_negotiation.cc:407] Connection header received
0923 22:11:34.111903 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.111904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.111914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.111985 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:34.113017 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.113804 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.114884 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.115106 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.115185 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.117435 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:34.117437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:34.117437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:34.117444 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:34.118680 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:34.119053 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:34.119054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:34.119054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:34.119102 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:34.119319 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:34.119320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:34.119320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:34.119425 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:34.119475 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.119806 (+ 331us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.119867 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:11:34.120168 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.111032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41733 (local address 127.1.244.195:46983)
0923 22:11:34.111197 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.111239 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.111810 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.112042 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.112046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.112626 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:34.112845 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.112878 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.113902 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.113903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.114768 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.114771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.115206 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.115207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.115339 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.115926 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:34.115935 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:34.117346 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:34.118749 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:34.118755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:34.118757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:34.118984 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:34.119164 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:34.119165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:34.119165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:34.119265 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:34.119493 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:34.119497 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:34.119690 (+ 193us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.119779 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.119906 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260923 22:11:34.120669 20528 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41733
I20260923 22:11:34.120783 20528 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:34.120963 20528 heartbeater.cc:511] Master 127.1.244.254:41733 requested a full tablet report, sending...
I20260923 22:11:34.121344 20069 ts_manager.cc:194] Registered new tserver with Master: a75d87bfceaa42718657368a62685709 (127.1.244.195:44001)
I20260923 22:11:34.121727 20069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:46983
I20260923 22:11:34.124915 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:34.132475 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.126285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.1.244.254:41733)
0923 22:11:34.126349 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.126351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.126471 (+ 120us) server_negotiation.cc:407] Connection header received
0923 22:11:34.126523 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.126523 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.126531 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.126570 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:34.127118 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.127659 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.128375 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.128556 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.128612 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.129916 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:34.129917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:34.129917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:34.129923 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:34.131215 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:34.131660 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:34.131661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:34.131661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:34.131712 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:34.131928 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:34.131929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:34.131929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:34.132057 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:34.132136 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.132353 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.132408 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:11:34.134086 20069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:34.135710 20069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:34.142896 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.138415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44320 (local address 127.1.244.195:44001)
0923 22:11:34.138620 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.138625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.138952 (+ 327us) server_negotiation.cc:407] Connection header received
0923 22:11:34.139191 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.139195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.139211 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.139266 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:34.139500 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.140237 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.141975 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.142458 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.142562 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.142621 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.142770 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.142812 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":102,"threads_started":1}
I20260923 22:11:34.142918 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.138156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:44001 (local address 127.0.0.1:44320)
0923 22:11:34.138888 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.138898 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.138951 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.139377 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.139378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.139379 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:34.139416 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.139417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.140369 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.140369 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.141874 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.141875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.142567 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.142568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.142650 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.142665 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.142735 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.142794 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":36,"thread_start_us":87,"threads_started":1}
I20260923 22:11:34.143085 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.138427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.1.244.193:41047)
0923 22:11:34.138695 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.138698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.139087 (+ 389us) server_negotiation.cc:407] Connection header received
0923 22:11:34.139141 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.139144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.139160 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.139239 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:34.139351 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.140431 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.141730 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.142477 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.142555 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.142621 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.142960 (+ 339us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.143005 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":114,"threads_started":1}
I20260923 22:11:34.143085 20542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.138231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37406 (local address 127.1.244.194:42987)
0923 22:11:34.138632 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.138636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.138645 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:11:34.138677 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.138679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.138691 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.138787 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:34.138996 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.140209 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.141861 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.142536 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.142594 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.142639 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.142970 (+ 331us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.143008 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":77,"threads_started":1}
I20260923 22:11:34.143131 20539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.138152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42987 (local address 127.0.0.1:37406)
0923 22:11:34.138325 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.138368 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.138436 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.138792 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.138796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.138799 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:34.138898 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.138901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.140336 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.140337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.141769 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.141772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.142668 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.142668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.142729 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.142736 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.142786 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.142849 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":73,"thread_start_us":62,"threads_started":1}
I20260923 22:11:34.143213 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.138156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41047 (local address 127.0.0.1:55484)
0923 22:11:34.139026 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.139034 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.139081 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.139250 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.139250 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.139252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:34.139295 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.139297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.140541 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.140542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.141638 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.141641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.142660 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.142661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.142729 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.142736 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.142786 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.142849 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":9,"thread_start_us":87,"threads_started":1}
I20260923 22:11:34.144008 20194 tablet_service.cc:1467] Processing CreateTablet for tablet a3852bb35bcc48c58904368ce6e8ca27 (DEFAULT_TABLE table=test-table [id=686a984e45f44fdc80857f47a5dc8565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:34.143966 20328 tablet_service.cc:1467] Processing CreateTablet for tablet a3852bb35bcc48c58904368ce6e8ca27 (DEFAULT_TABLE table=test-table [id=686a984e45f44fdc80857f47a5dc8565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:34.144383 20194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3852bb35bcc48c58904368ce6e8ca27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:34.144388 20328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3852bb35bcc48c58904368ce6e8ca27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:34.144570 20462 tablet_service.cc:1467] Processing CreateTablet for tablet a3852bb35bcc48c58904368ce6e8ca27 (DEFAULT_TABLE table=test-table [id=686a984e45f44fdc80857f47a5dc8565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:34.144876 20462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3852bb35bcc48c58904368ce6e8ca27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:34.147507 20545 tablet_bootstrap.cc:492] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90: Bootstrap starting.
I20260923 22:11:34.148062 20547 tablet_bootstrap.cc:492] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709: Bootstrap starting.
I20260923 22:11:34.148376 20545 tablet_bootstrap.cc:654] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:34.148730 20547 tablet_bootstrap.cc:654] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:34.148959 20545 log.cc:824] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:34.149256 20547 log.cc:824] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:34.149576 20546 tablet_bootstrap.cc:492] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: Bootstrap starting.
I20260923 22:11:34.149709 20545 tablet_bootstrap.cc:492] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90: No bootstrap required, opened a new log
I20260923 22:11:34.149780 20545 ts_tablet_manager.cc:1397] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:11:34.150156 20546 tablet_bootstrap.cc:654] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:34.150628 20546 log.cc:824] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:34.151127 20547 tablet_bootstrap.cc:492] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709: No bootstrap required, opened a new log
I20260923 22:11:34.151203 20547 ts_tablet_manager.cc:1397] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:11:34.151420 20546 tablet_bootstrap.cc:492] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: No bootstrap required, opened a new log
I20260923 22:11:34.151489 20546 ts_tablet_manager.cc:1397] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:34.151607 20545 raft_consensus.cc:348] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } }
I20260923 22:11:34.151798 20545 raft_consensus.cc:374] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:34.151844 20545 raft_consensus.cc:729] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc2cdb4a591a4f5ca518f35da92ead90, State: Initialized, Role: FOLLOWER
I20260923 22:11:34.151996 20545 consensus_queue.cc:260] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } }
I20260923 22:11:34.152272 20545 ts_tablet_manager.cc:1428] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:11:34.152420 20260 heartbeater.cc:503] Master 127.1.244.254:41733 was elected leader, sending a full tablet report...
I20260923 22:11:34.153254 20547 raft_consensus.cc:348] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } }
I20260923 22:11:34.153430 20547 raft_consensus.cc:374] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:34.153467 20547 raft_consensus.cc:729] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75d87bfceaa42718657368a62685709, State: Initialized, Role: FOLLOWER
I20260923 22:11:34.153517 20546 raft_consensus.cc:348] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } }
I20260923 22:11:34.153604 20547 consensus_queue.cc:260] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } }
I20260923 22:11:34.153683 20546 raft_consensus.cc:374] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:34.153723 20546 raft_consensus.cc:729] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a5b0afea7d459a84f5b4ec8e817f04, State: Initialized, Role: FOLLOWER
I20260923 22:11:34.153844 20546 consensus_queue.cc:260] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } }
I20260923 22:11:34.153893 20547 ts_tablet_manager.cc:1428] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:11:34.153908 20528 heartbeater.cc:503] Master 127.1.244.254:41733 was elected leader, sending a full tablet report...
I20260923 22:11:34.154099 20546 ts_tablet_manager.cc:1428] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:11:34.154127 20394 heartbeater.cc:503] Master 127.1.244.254:41733 was elected leader, sending a full tablet report...
W20260923 22:11:34.185513 20395 tablet.cc:2369] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:11:34.264307 20261 tablet.cc:2369] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:34.331498 20552 raft_consensus.cc:482] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:34.331715 20552 raft_consensus.cc:504] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } }
I20260923 22:11:34.332141 20552 leader_election.cc:288] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc2cdb4a591a4f5ca518f35da92ead90 (127.1.244.193:41047), 37a5b0afea7d459a84f5b4ec8e817f04 (127.1.244.194:42987)
I20260923 22:11:34.335001 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.332324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41047 (local address 127.1.244.195:38851)
0923 22:11:34.332401 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.332415 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.332483 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.332644 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.332645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.332648 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:34.332721 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.332731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.333390 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.333391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.334133 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.334135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.334701 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.334702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.334791 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.334794 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.334880 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.334926 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 22:11:34.335001 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.332434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:38851 (local address 127.1.244.193:41047)
0923 22:11:34.332489 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.332492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.332507 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:11:34.332543 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.332545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.332555 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.332620 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:34.332810 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.333295 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.334233 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.334616 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.334706 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.334714 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.334892 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.334936 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:11:34.335318 20214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3852bb35bcc48c58904368ce6e8ca27" candidate_uuid: "a75d87bfceaa42718657368a62685709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" is_pre_election: true
I20260923 22:11:34.335490 20542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.332491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:47743 (local address 127.1.244.194:42987)
0923 22:11:34.332540 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.332542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.332697 (+ 155us) server_negotiation.cc:407] Connection header received
0923 22:11:34.332749 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.332750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.332759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.332835 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:34.333035 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.333547 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.334593 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.335010 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.335110 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.335122 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.335364 (+ 242us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.335410 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:11:34.335489 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.332416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42987 (local address 127.1.244.195:47743)
0923 22:11:34.332595 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.332604 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.332655 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.332839 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.332840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.332842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:34.332921 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.332922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.333641 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.333642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.334455 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.334457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.335188 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.335189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.335275 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.335279 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.335358 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.335400 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":128,"threads_started":1}
I20260923 22:11:34.335502 20214 raft_consensus.cc:2454] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a75d87bfceaa42718657368a62685709 in term 0.
I20260923 22:11:34.335865 20418 leader_election.cc:302] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a75d87bfceaa42718657368a62685709, fc2cdb4a591a4f5ca518f35da92ead90; no voters:
I20260923 22:11:34.335873 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3852bb35bcc48c58904368ce6e8ca27" candidate_uuid: "a75d87bfceaa42718657368a62685709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" is_pre_election: true
I20260923 22:11:34.336032 20348 raft_consensus.cc:2454] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a75d87bfceaa42718657368a62685709 in term 0.
I20260923 22:11:34.336036 20552 raft_consensus.cc:2781] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:34.336102 20552 raft_consensus.cc:482] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:34.336122 20552 raft_consensus.cc:3037] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:34.336959 20552 raft_consensus.cc:504] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } }
I20260923 22:11:34.337123 20552 leader_election.cc:288] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [CANDIDATE]: Term 1 election: Requested vote from peers fc2cdb4a591a4f5ca518f35da92ead90 (127.1.244.193:41047), 37a5b0afea7d459a84f5b4ec8e817f04 (127.1.244.194:42987)
I20260923 22:11:34.337271 20214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3852bb35bcc48c58904368ce6e8ca27" candidate_uuid: "a75d87bfceaa42718657368a62685709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc2cdb4a591a4f5ca518f35da92ead90"
I20260923 22:11:34.337365 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3852bb35bcc48c58904368ce6e8ca27" candidate_uuid: "a75d87bfceaa42718657368a62685709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a5b0afea7d459a84f5b4ec8e817f04"
I20260923 22:11:34.337374 20214 raft_consensus.cc:3037] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:34.337450 20348 raft_consensus.cc:3037] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:34.338107 20214 raft_consensus.cc:2454] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a75d87bfceaa42718657368a62685709 in term 1.
I20260923 22:11:34.338259 20348 raft_consensus.cc:2454] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a75d87bfceaa42718657368a62685709 in term 1.
I20260923 22:11:34.338302 20418 leader_election.cc:302] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a75d87bfceaa42718657368a62685709, fc2cdb4a591a4f5ca518f35da92ead90; no voters:
I20260923 22:11:34.338416 20552 raft_consensus.cc:2781] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:34.338575 20552 raft_consensus.cc:686] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 1 LEADER]: Becoming Leader. State: Replica: a75d87bfceaa42718657368a62685709, State: Running, Role: LEADER
I20260923 22:11:34.338672 20552 consensus_queue.cc:237] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } }
I20260923 22:11:34.339638 20069 catalog_manager.cc:5495] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 reported cstate change: term changed from 0 to 1, leader changed from <none> to a75d87bfceaa42718657368a62685709 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "a75d87bfceaa42718657368a62685709" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75d87bfceaa42718657368a62685709" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44001 } health_report { overall_health: HEALTHY } } }
W20260923 22:11:34.361011 20529 tablet.cc:2369] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:34.371946 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.369495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.1.244.195:44001)
0923 22:11:34.369548 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.369551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.369578 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:11:34.369615 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.369616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.369634 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.369687 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:34.369869 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.370325 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.371066 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.371257 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.371311 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.371459 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:34.371578 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:34.371655 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.371776 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.371785 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:11:34.374038 20214 raft_consensus.cc:1261] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [term 1 FOLLOWER]: Refusing update from remote peer a75d87bfceaa42718657368a62685709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:34.374032 20348 raft_consensus.cc:1261] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [term 1 FOLLOWER]: Refusing update from remote peer a75d87bfceaa42718657368a62685709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:34.374459 20552 consensus_queue.cc:1035] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc2cdb4a591a4f5ca518f35da92ead90" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:34.374635 20555 consensus_queue.cc:1035] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37a5b0afea7d459a84f5b4ec8e817f04" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:34.379050 20559 mvcc.cc:205] Tried to move back new op lower bound from 7332665320952614912 to 7332665320812085248. Current Snapshot: MvccSnapshot[applied={T|T < 7332665320952614912 or (T in {7332665320952614912})}]
I20260923 22:11:34.379380 20069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:34.379482 20069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:34.380368 20069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1346733eb5cb4d02b2c7f912f932ff03: Sending DeleteTablet for 3 replicas of tablet a3852bb35bcc48c58904368ce6e8ca27
I20260923 22:11:34.380716 20194 tablet_service.cc:1514] Processing DeleteTablet for tablet a3852bb35bcc48c58904368ce6e8ca27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:34 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:55484
I20260923 22:11:34.380822 20328 tablet_service.cc:1514] Processing DeleteTablet for tablet a3852bb35bcc48c58904368ce6e8ca27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:34 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:37406
I20260923 22:11:34.380790 20462 tablet_service.cc:1514] Processing DeleteTablet for tablet a3852bb35bcc48c58904368ce6e8ca27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:34 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44320
I20260923 22:11:34.380992 20568 tablet_replica.cc:330] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90: stopping tablet replica
I20260923 22:11:34.381100 20569 tablet_replica.cc:330] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: stopping tablet replica
I20260923 22:11:34.381102 20568 raft_consensus.cc:2229] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:34.381178 20569 raft_consensus.cc:2229] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:34.381211 20568 raft_consensus.cc:2258] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:34.381255 20569 raft_consensus.cc:2258] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:34.381390 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 20131
I20260923 22:11:34.381646 20568 ts_tablet_manager.cc:1905] T a3852bb35bcc48c58904368ce6e8ca27 P fc2cdb4a591a4f5ca518f35da92ead90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:34.381764 20569 ts_tablet_manager.cc:1905] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:34.383435 20569 ts_tablet_manager.cc:1918] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:34.383490 20569 log.cc:1196] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/a3852bb35bcc48c58904368ce6e8ca27
I20260923 22:11:34.383798 20569 ts_tablet_manager.cc:1939] T a3852bb35bcc48c58904368ce6e8ca27 P 37a5b0afea7d459a84f5b4ec8e817f04: Deleting consensus metadata
I20260923 22:11:34.384196 20055 catalog_manager.cc:4841] TS 37a5b0afea7d459a84f5b4ec8e817f04 (127.1.244.194:42987): tablet a3852bb35bcc48c58904368ce6e8ca27 (table test-table [id=686a984e45f44fdc80857f47a5dc8565]) successfully deleted
W20260923 22:11:34.387440 20056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:41047 (error 108)
I20260923 22:11:34.387722 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 20265
I20260923 22:11:34.388383 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.387861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41047 (local address 127.0.0.1:55498)
0923 22:11:34.388302 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.388343 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418}
W20260923 22:11:34.389681 20056 catalog_manager.cc:4584] TS fc2cdb4a591a4f5ca518f35da92ead90 (127.1.244.193:41047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a3852bb35bcc48c58904368ce6e8ca27: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41047: connect: Connection refused (error 111)
I20260923 22:11:34.392901 20570 tablet_replica.cc:330] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709: stopping tablet replica
I20260923 22:11:34.393046 20570 raft_consensus.cc:2229] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:11:34.393133 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 20399
I20260923 22:11:34.393195 20570 raft_consensus.cc:2258] T a3852bb35bcc48c58904368ce6e8ca27 P a75d87bfceaa42718657368a62685709 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:34.398497 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 20037
I20260923 22:11:34.398671 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.398510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:44001 (local address 127.0.0.1:44338)
0923 22:11:34.398590 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.398628 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.195:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
2026-09-23T22:11:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2109 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-23T22:11:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:34Z Disabled control of system clock
I20260923 22:11:34.427302 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37779
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:37779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:34.517342 20575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:34.517575 20575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:34.517596 20575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:34.518707 20575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:34.518743 20575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:34.518755 20575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:34.518767 20575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:34.518780 20575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:34.520094 20575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:37779
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:34.520295 20575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:34.520532 20575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:34.525444 20580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:34.525478 20583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:34.525475 20581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:34.525599 20575 server_base.cc:1034] running on GCE node
I20260923 22:11:34.526010 20575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:34.526314 20575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:34.527487 20575 hybrid_clock.cc:648] HybridClock initialized: now 1790201494527465 us; error 44 us; skew 500 ppm
I20260923 22:11:34.534138 20575 webserver.cc:460] Webserver started at http://127.1.244.254:44263/ using document root <none> and password file <none>
I20260923 22:11:34.534436 20575 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:34.534513 20575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:34.534669 20575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:34.535856 20575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "e53f19c0ac4244f7b657028b19bde9a9"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "9eba0c1060a53afeeda093a600292c99"
server_key_iv: "20da9f186e7a7a9e396dadc15e881136"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.536226 20575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "e53f19c0ac4244f7b657028b19bde9a9"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "9eba0c1060a53afeeda093a600292c99"
server_key_iv: "20da9f186e7a7a9e396dadc15e881136"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.537655 20575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:34.538507 20589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.538720 20575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:34.538852 20575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "e53f19c0ac4244f7b657028b19bde9a9"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "9eba0c1060a53afeeda093a600292c99"
server_key_iv: "20da9f186e7a7a9e396dadc15e881136"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.538935 20575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:34.551741 20575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:34.552090 20575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:34.552253 20575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:34.556424 20575 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:37779
I20260923 22:11:34.556463 20641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:37779 every 8 connection(s)
I20260923 22:11:34.556836 20575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:34.557991 20642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:34.560583 20642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9: Bootstrap starting.
I20260923 22:11:34.561249 20642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:34.561589 20642 log.cc:824] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:34.562345 20642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9: No bootstrap required, opened a new log
I20260923 22:11:34.562364 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 20575
I20260923 22:11:34.562475 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:34.562664 2003 external_mini_cluster.cc:1428] Setting key b490263a4a8f10d4c78ab98c2a0306b3
I20260923 22:11:34.564677 20642 raft_consensus.cc:348] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e53f19c0ac4244f7b657028b19bde9a9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37779 } }
I20260923 22:11:34.564875 20642 raft_consensus.cc:374] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:34.564921 20642 raft_consensus.cc:729] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53f19c0ac4244f7b657028b19bde9a9, State: Initialized, Role: FOLLOWER
I20260923 22:11:34.565076 20642 consensus_queue.cc:260] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e53f19c0ac4244f7b657028b19bde9a9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37779 } }
I20260923 22:11:34.565181 20642 raft_consensus.cc:388] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:34.565219 20642 raft_consensus.cc:482] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:34.565272 20642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:34.565876 20642 raft_consensus.cc:504] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e53f19c0ac4244f7b657028b19bde9a9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37779 } }
I20260923 22:11:34.565969 20642 leader_election.cc:302] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e53f19c0ac4244f7b657028b19bde9a9; no voters:
I20260923 22:11:34.566186 20642 leader_election.cc:288] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:34.566236 20647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:34.566375 20647 raft_consensus.cc:686] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [term 1 LEADER]: Becoming Leader. State: Replica: e53f19c0ac4244f7b657028b19bde9a9, State: Running, Role: LEADER
I20260923 22:11:34.566462 20647 consensus_queue.cc:237] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e53f19c0ac4244f7b657028b19bde9a9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37779 } }
I20260923 22:11:34.566493 20642 sys_catalog.cc:564] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:34.566798 20648 sys_catalog.cc:455] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e53f19c0ac4244f7b657028b19bde9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e53f19c0ac4244f7b657028b19bde9a9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37779 } } }
I20260923 22:11:34.567044 20648 sys_catalog.cc:458] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:34.566797 20647 sys_catalog.cc:455] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e53f19c0ac4244f7b657028b19bde9a9. Latest consensus state: current_term: 1 leader_uuid: "e53f19c0ac4244f7b657028b19bde9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e53f19c0ac4244f7b657028b19bde9a9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37779 } } }
I20260923 22:11:34.567579 20647 sys_catalog.cc:458] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:34.567767 20654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:34.568315 20654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:34.569898 20654 catalog_manager.cc:1347] Generated new cluster ID: 51f205f7c2cb4f88b469e629560dad60
I20260923 22:11:34.569953 20654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:34.570572 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.563264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.1.244.254:37779)
0923 22:11:34.563463 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.563470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.563502 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:11:34.564662 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.564670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.564738 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.564906 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:34.565207 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.566896 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.568051 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.568642 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.568771 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.569640 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:34.569644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:34.569644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:34.570034 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:34.570088 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:34.570104 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:34.570183 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.570286 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.570367 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260923 22:11:34.578262 20654 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:34.578751 20654 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:34.586714 20654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9: Generated new TSK 0
I20260923 22:11:34.586951 20654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:34.591850 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37779
--builtin_ntp_servers=127.1.244.212:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:34.675194 20666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:34.675452 20666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:34.675496 20666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:34.676795 20666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:34.676862 20666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:34.676906 20666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:11:34.678279 20666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:34.678570 20666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:34.678885 20666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:34.684310 20672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:34.684358 20674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:34.684576 20671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:34.684504 20666 server_base.cc:1034] running on GCE node
I20260923 22:11:34.684831 20666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:34.685168 20666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:34.686340 20666 hybrid_clock.cc:648] HybridClock initialized: now 1790201494686320 us; error 41 us; skew 500 ppm
I20260923 22:11:34.687785 20666 webserver.cc:460] Webserver started at http://127.1.244.193:46827/ using document root <none> and password file <none>
I20260923 22:11:34.688066 20666 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:34.688114 20666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:34.688287 20666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:34.689396 20666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "66a0731255c4422481174f844dbab8f5"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "e5fe25f1e3a62dca80aab5092cb16e52"
server_key_iv: "6cf8c92daf942a0812c913b1752eca3c"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.689782 20666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "66a0731255c4422481174f844dbab8f5"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "e5fe25f1e3a62dca80aab5092cb16e52"
server_key_iv: "6cf8c92daf942a0812c913b1752eca3c"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.691141 20666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:34.691987 20680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.692142 20666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.692251 20666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "66a0731255c4422481174f844dbab8f5"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "e5fe25f1e3a62dca80aab5092cb16e52"
server_key_iv: "6cf8c92daf942a0812c913b1752eca3c"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.692330 20666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:34.707628 20666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:34.707957 20666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:34.708112 20666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:34.708359 20666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:34.708714 20666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:34.708750 20666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.708772 20666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:34.708822 20666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.714679 20666 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:35413
I20260923 22:11:34.714741 20793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:35413 every 8 connection(s)
I20260923 22:11:34.715122 20666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:34.717006 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 20666
I20260923 22:11:34.717115 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:34.717294 2003 external_mini_cluster.cc:1428] Setting key cfd40fdbc98c07e0aa809f23069b4478
I20260923 22:11:34.718339 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37779
--builtin_ntp_servers=127.1.244.212:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:11:34.721802 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.716002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:41231 (local address 127.1.244.254:37779)
0923 22:11:34.716094 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.716097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.716317 (+ 220us) server_negotiation.cc:407] Connection header received
0923 22:11:34.716852 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.716853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.716864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.716928 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:34.717921 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.718591 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.720142 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.720491 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.720576 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.721500 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:34.721501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:34.721501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:34.721509 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:34.721526 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:34.721530 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:34.721581 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.721691 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.721724 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 22:11:34.722141 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.715971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37779 (local address 127.1.244.193:41231)
0923 22:11:34.716178 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.716220 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.716717 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.717055 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.717060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.717072 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:34.717820 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.717827 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.718688 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.718690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.720048 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.720052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.720586 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.720587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.720699 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.721334 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:34.721335 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:34.721364 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:34.721389 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:34.721599 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:34.721605 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:34.721608 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.721677 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.721785 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260923 22:11:34.722818 20794 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37779
I20260923 22:11:34.722903 20794 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:34.723109 20794 heartbeater.cc:511] Master 127.1.244.254:37779 requested a full tablet report, sending...
I20260923 22:11:34.723723 20606 ts_manager.cc:194] Registered new tserver with Master: 66a0731255c4422481174f844dbab8f5 (127.1.244.193:35413)
I20260923 22:11:34.724326 20606 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.193:41231
W20260923 22:11:34.802363 20797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:34.802601 20797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:34.802639 20797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:34.804018 20797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:34.804087 20797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:34.804132 20797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:11:34.805541 20797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:34.805830 20797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:34.806135 20797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:34.811300 20805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:34.811307 20803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:34.811324 20802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:34.811677 20797 server_base.cc:1034] running on GCE node
I20260923 22:11:34.811918 20797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:34.812158 20797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:34.813341 20797 hybrid_clock.cc:648] HybridClock initialized: now 1790201494813322 us; error 49 us; skew 500 ppm
I20260923 22:11:34.814630 20797 webserver.cc:460] Webserver started at http://127.1.244.194:34005/ using document root <none> and password file <none>
I20260923 22:11:34.814889 20797 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:34.814965 20797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:34.815120 20797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:34.816324 20797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "e333eaff2df9d79222651b0f5781fe5a"
server_key_iv: "42f3d2d58d18ef7db0af2249b99fc8e7"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.816700 20797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "e333eaff2df9d79222651b0f5781fe5a"
server_key_iv: "42f3d2d58d18ef7db0af2249b99fc8e7"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.818053 20797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:11:34.818889 20811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.819087 20797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:34.819198 20797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "e333eaff2df9d79222651b0f5781fe5a"
server_key_iv: "42f3d2d58d18ef7db0af2249b99fc8e7"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.819288 20797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:34.833644 20797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:34.833982 20797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:34.834139 20797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:34.834372 20797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:34.834746 20797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:34.834805 20797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.834846 20797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:34.834873 20797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.840860 20797 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:42725
I20260923 22:11:34.840919 20924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:42725 every 8 connection(s)
I20260923 22:11:34.841312 20797 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:34.843946 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 20797
I20260923 22:11:34.844067 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:34.844270 2003 external_mini_cluster.cc:1428] Setting key c919c0d507d3fdb8084f31257dabd470
I20260923 22:11:34.845518 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37779
--builtin_ntp_servers=127.1.244.212:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:11:34.848003 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.842119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:42885 (local address 127.1.244.254:37779)
0923 22:11:34.842184 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.842186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.842420 (+ 234us) server_negotiation.cc:407] Connection header received
0923 22:11:34.842950 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.842951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.842966 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.843023 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:34.843757 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.844355 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.845770 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.846110 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.846154 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.847626 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:34.847628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:34.847628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:34.847633 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:34.847651 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:34.847656 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:34.847714 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.847870 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.847904 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:11:34.848242 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.842095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37779 (local address 127.1.244.194:42885)
0923 22:11:34.842285 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.842320 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.842801 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.843082 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.843085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.843094 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:34.843671 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.843677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.844454 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.844456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.845667 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.845671 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.846248 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.846249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.846420 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.847104 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:34.847105 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:34.847172 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:34.847202 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:34.847742 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:34.847750 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:34.847754 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.847860 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.847990 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260923 22:11:34.848953 20925 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37779
I20260923 22:11:34.849066 20925 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:34.849277 20925 heartbeater.cc:511] Master 127.1.244.254:37779 requested a full tablet report, sending...
I20260923 22:11:34.849653 20606 ts_manager.cc:194] Registered new tserver with Master: a3c90646cb1f4ec5a0c6b50fe6a0ccf2 (127.1.244.194:42725)
I20260923 22:11:34.850203 20606 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.194:42885
W20260923 22:11:34.933705 20928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:34.933954 20928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:34.933977 20928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:34.935143 20928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:34.935184 20928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:34.935210 20928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:11:34.936638 20928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:34.936918 20928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:34.937248 20928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:34.942569 20936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:34.942692 20933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:34.942569 20934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:34.943044 20928 server_base.cc:1034] running on GCE node
I20260923 22:11:34.943226 20928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:34.943442 20928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:34.944595 20928 hybrid_clock.cc:648] HybridClock initialized: now 1790201494944576 us; error 33 us; skew 500 ppm
I20260923 22:11:34.946092 20928 webserver.cc:460] Webserver started at http://127.1.244.195:42961/ using document root <none> and password file <none>
I20260923 22:11:34.946367 20928 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:34.946444 20928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:34.946607 20928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:34.947877 20928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "1fd1c4f933934d89a2babc5693469c59"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "8887bb7806d1f2234ef95efbe8d34a87"
server_key_iv: "8a310630bcb313e609c32454a25c0087"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.948261 20928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "1fd1c4f933934d89a2babc5693469c59"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "8887bb7806d1f2234ef95efbe8d34a87"
server_key_iv: "8a310630bcb313e609c32454a25c0087"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.956969 20928 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
I20260923 22:11:34.958310 20942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.958539 20928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:34.958650 20928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "1fd1c4f933934d89a2babc5693469c59"
format_stamp: "Formatted at 2026-09-23 22:11:34 on dist-test-slave-b5jz"
server_key: "8887bb7806d1f2234ef95efbe8d34a87"
server_key_iv: "8a310630bcb313e609c32454a25c0087"
server_key_version: "encryptionkey@0"
I20260923 22:11:34.958753 20928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:34.998234 20928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:34.998610 20928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:34.998785 20928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:34.999047 20928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:34.999413 20928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:34.999473 20928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:34.999517 20928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:34.999571 20928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.006476 20928 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:40891
I20260923 22:11:35.006515 21055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:40891 every 8 connection(s)
I20260923 22:11:35.006879 20928 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:35.012766 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.007698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:52527 (local address 127.1.244.254:37779)
0923 22:11:35.007771 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.007774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.008039 (+ 265us) server_negotiation.cc:407] Connection header received
0923 22:11:35.008533 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.008534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.008545 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.008602 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:35.009278 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.009771 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.010743 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.010937 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.010997 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.012515 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:35.012516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:35.012516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:35.012521 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:35.012536 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:35.012540 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:35.012585 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.012670 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.012693 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:11:35.012989 21058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.007706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37779 (local address 127.1.244.195:52527)
0923 22:11:35.007911 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.007949 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.008442 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.008656 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.008660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.008669 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:35.009182 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.009187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.009855 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.009855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.010657 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.010660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.011017 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.011018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.011127 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.012369 (+ 1242us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:35.012370 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:35.012400 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:35.012425 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:35.012598 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:35.012602 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:35.012604 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.012656 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.012752 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260923 22:11:35.013415 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 20928
I20260923 22:11:35.013538 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:35.013548 21056 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37779
I20260923 22:11:35.013636 21056 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:35.013741 2003 external_mini_cluster.cc:1428] Setting key a2ad91522cfbd80964d374d1c2f960ad
I20260923 22:11:35.013877 21056 heartbeater.cc:511] Master 127.1.244.254:37779 requested a full tablet report, sending...
I20260923 22:11:35.014261 20606 ts_manager.cc:194] Registered new tserver with Master: 1fd1c4f933934d89a2babc5693469c59 (127.1.244.195:40891)
I20260923 22:11:35.014683 20606 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.195:52527
I20260923 22:11:35.014753 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:35.019111 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.016431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51654 (local address 127.1.244.254:37779)
0923 22:11:35.016491 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.016492 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.016532 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:11:35.016591 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.016592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.016599 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.016656 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:35.016847 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.017310 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.018036 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.018206 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.018256 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.018878 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:35.018879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:35.018879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:35.018883 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:35.018894 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:35.018897 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:35.018965 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.019050 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.019058 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:11:35.020955 20606 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:35.022496 20606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:35.029068 21068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.025031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.1.244.195:40891)
0923 22:11:35.025235 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.025240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.025360 (+ 120us) server_negotiation.cc:407] Connection header received
0923 22:11:35.025473 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.025476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.025494 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.025578 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:35.025763 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.026632 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.028103 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.028670 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.028767 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.028875 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.028936 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.028974 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260923 22:11:35.029057 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.024827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40891 (local address 127.0.0.1:54822)
0923 22:11:35.025212 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.025255 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.025375 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.025591 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.025594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.025597 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:35.025677 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.025679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.026725 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.026726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.028007 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.028010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.028766 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.028767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.028844 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.028857 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.028924 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.028979 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":76,"threads_started":1}
I20260923 22:11:35.029479 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.024899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:35413 (local address 127.0.0.1:47598)
0923 22:11:35.025433 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.025444 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.025503 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.025698 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.025699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.025701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:35.025749 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.025751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.027087 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.027087 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.028178 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.028180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.029219 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.029220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.029296 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.029299 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.029363 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.029402 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":99,"threads_started":1}
I20260923 22:11:35.029537 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.025029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47598 (local address 127.1.244.193:35413)
0923 22:11:35.025206 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.025212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.025500 (+ 288us) server_negotiation.cc:407] Connection header received
0923 22:11:35.025593 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.025595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.025611 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.025688 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:35.025846 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.026993 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.028268 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.029012 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.029084 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.029175 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.029409 (+ 234us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.029458 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260923 22:11:35.029848 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.025045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42725 (local address 127.0.0.1:58904)
0923 22:11:35.025605 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.025614 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.025824 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.026019 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.026020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.026022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:35.026058 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.026059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.027279 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.027280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.028201 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.028203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.029468 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.029469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.029663 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.029691 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.029746 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.029791 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":76,"threads_started":1}
I20260923 22:11:35.029866 20990 tablet_service.cc:1467] Processing CreateTablet for tablet da60d80da02046ba817ec996a825f72b (DEFAULT_TABLE table=test-table [id=753a412f76df490ab9028f29cc972325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:35.030251 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.025246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.1.244.194:42725)
0923 22:11:35.025427 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.025431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.025879 (+ 448us) server_negotiation.cc:407] Connection header received
0923 22:11:35.025926 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.025928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.025944 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.026017 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:35.026122 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.027185 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.028318 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.029014 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.029084 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.030093 (+ 1009us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.030123 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.030165 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260923 22:11:35.030340 20990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da60d80da02046ba817ec996a825f72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:35.030988 20859 tablet_service.cc:1467] Processing CreateTablet for tablet da60d80da02046ba817ec996a825f72b (DEFAULT_TABLE table=test-table [id=753a412f76df490ab9028f29cc972325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:35.031343 20859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da60d80da02046ba817ec996a825f72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:35.032150 20728 tablet_service.cc:1467] Processing CreateTablet for tablet da60d80da02046ba817ec996a825f72b (DEFAULT_TABLE table=test-table [id=753a412f76df490ab9028f29cc972325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:35.032486 20728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da60d80da02046ba817ec996a825f72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:35.034101 21070 tablet_bootstrap.cc:492] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: Bootstrap starting.
I20260923 22:11:35.034812 21071 tablet_bootstrap.cc:492] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Bootstrap starting.
I20260923 22:11:35.034881 21070 tablet_bootstrap.cc:654] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:35.035215 21070 log.cc:824] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:35.035435 21071 tablet_bootstrap.cc:654] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:35.035920 21071 log.cc:824] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:35.036056 21070 tablet_bootstrap.cc:492] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: No bootstrap required, opened a new log
I20260923 22:11:35.036132 21070 ts_tablet_manager.cc:1397] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:35.036118 21073 tablet_bootstrap.cc:492] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5: Bootstrap starting.
I20260923 22:11:35.036652 21073 tablet_bootstrap.cc:654] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:35.036945 21073 log.cc:824] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:35.037635 21073 tablet_bootstrap.cc:492] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5: No bootstrap required, opened a new log
I20260923 22:11:35.037730 21073 ts_tablet_manager.cc:1397] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:35.038232 21070 raft_consensus.cc:348] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } }
I20260923 22:11:35.038384 21070 raft_consensus.cc:374] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:35.038406 21070 raft_consensus.cc:729] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd1c4f933934d89a2babc5693469c59, State: Initialized, Role: FOLLOWER
I20260923 22:11:35.038537 21070 consensus_queue.cc:260] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } }
I20260923 22:11:35.038851 21070 ts_tablet_manager.cc:1428] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:35.038889 21056 heartbeater.cc:503] Master 127.1.244.254:37779 was elected leader, sending a full tablet report...
I20260923 22:11:35.040500 21073 raft_consensus.cc:348] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } }
I20260923 22:11:35.040696 21073 raft_consensus.cc:374] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:35.040738 21073 raft_consensus.cc:729] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a0731255c4422481174f844dbab8f5, State: Initialized, Role: FOLLOWER
I20260923 22:11:35.040869 21073 consensus_queue.cc:260] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } }
I20260923 22:11:35.041170 21073 ts_tablet_manager.cc:1428] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260923 22:11:35.041332 20794 heartbeater.cc:503] Master 127.1.244.254:37779 was elected leader, sending a full tablet report...
I20260923 22:11:35.042353 21071 tablet_bootstrap.cc:492] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: No bootstrap required, opened a new log
I20260923 22:11:35.042436 21071 ts_tablet_manager.cc:1397] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260923 22:11:35.046903 21071 raft_consensus.cc:348] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } }
I20260923 22:11:35.047150 21071 raft_consensus.cc:374] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:35.047204 21071 raft_consensus.cc:729] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c90646cb1f4ec5a0c6b50fe6a0ccf2, State: Initialized, Role: FOLLOWER
I20260923 22:11:35.047351 21071 consensus_queue.cc:260] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } }
I20260923 22:11:35.047765 20925 heartbeater.cc:503] Master 127.1.244.254:37779 was elected leader, sending a full tablet report...
I20260923 22:11:35.048074 21071 ts_tablet_manager.cc:1428] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
W20260923 22:11:35.091952 20926 tablet.cc:2369] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:35.138455 21078 raft_consensus.cc:482] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:35.138659 21078 raft_consensus.cc:504] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } }
I20260923 22:11:35.139067 21078 leader_election.cc:288] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fd1c4f933934d89a2babc5693469c59 (127.1.244.195:40891), 66a0731255c4422481174f844dbab8f5 (127.1.244.193:35413)
I20260923 22:11:35.143187 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.139278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40891 (local address 127.1.244.194:58089)
0923 22:11:35.139347 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.139361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.139409 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.139680 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.139680 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.139682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:35.139787 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.139789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.140800 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.140801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.142093 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.142097 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.142870 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.142871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.142958 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.142963 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.143032 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.143081 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":65}
I20260923 22:11:35.143270 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.139394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:56325 (local address 127.1.244.193:35413)
0923 22:11:35.139458 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.139460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.139539 (+ 79us) server_negotiation.cc:407] Connection header received
0923 22:11:35.139655 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.139657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.139671 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.139729 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:35.139899 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.140646 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.142185 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.142747 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.142847 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.142857 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.143108 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.143202 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:11:35.143288 21068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.139376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:58089 (local address 127.1.244.195:40891)
0923 22:11:35.139450 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.139452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.139465 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:11:35.139537 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.139538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.139550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.139666 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:35.139897 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.140650 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.142185 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.142747 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.142847 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.142857 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.143180 (+ 323us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.143228 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:11:35.143193 21079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.139293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:35413 (local address 127.1.244.194:56325)
0923 22:11:35.139463 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.139473 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.139529 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.139751 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.139751 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.139753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:35.139820 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.139822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.140772 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.140772 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.142097 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.142098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.142870 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.142871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.142964 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.142967 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.143032 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.143081 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":30,"thread_start_us":97,"threads_started":1}
I20260923 22:11:35.143646 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da60d80da02046ba817ec996a825f72b" candidate_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd1c4f933934d89a2babc5693469c59" is_pre_election: true
I20260923 22:11:35.143863 21010 raft_consensus.cc:2454] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c90646cb1f4ec5a0c6b50fe6a0ccf2 in term 0.
I20260923 22:11:35.143934 20748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da60d80da02046ba817ec996a825f72b" candidate_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a0731255c4422481174f844dbab8f5" is_pre_election: true
I20260923 22:11:35.144083 20748 raft_consensus.cc:2454] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c90646cb1f4ec5a0c6b50fe6a0ccf2 in term 0.
I20260923 22:11:35.144296 20815 leader_election.cc:302] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd1c4f933934d89a2babc5693469c59, a3c90646cb1f4ec5a0c6b50fe6a0ccf2; no voters:
I20260923 22:11:35.144464 21078 raft_consensus.cc:2781] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:35.144531 21078 raft_consensus.cc:482] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:35.144562 21078 raft_consensus.cc:3037] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:35.145413 21078 raft_consensus.cc:504] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } }
I20260923 22:11:35.145565 21078 leader_election.cc:288] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [CANDIDATE]: Term 1 election: Requested vote from peers 1fd1c4f933934d89a2babc5693469c59 (127.1.244.195:40891), 66a0731255c4422481174f844dbab8f5 (127.1.244.193:35413)
I20260923 22:11:35.145704 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da60d80da02046ba817ec996a825f72b" candidate_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd1c4f933934d89a2babc5693469c59"
I20260923 22:11:35.145717 20748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da60d80da02046ba817ec996a825f72b" candidate_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a0731255c4422481174f844dbab8f5"
I20260923 22:11:35.145787 21010 raft_consensus.cc:3037] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:35.145828 20748 raft_consensus.cc:3037] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:35.146637 20748 raft_consensus.cc:2454] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c90646cb1f4ec5a0c6b50fe6a0ccf2 in term 1.
I20260923 22:11:35.146646 21010 raft_consensus.cc:2454] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c90646cb1f4ec5a0c6b50fe6a0ccf2 in term 1.
I20260923 22:11:35.146829 20812 leader_election.cc:302] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a0731255c4422481174f844dbab8f5, a3c90646cb1f4ec5a0c6b50fe6a0ccf2; no voters:
I20260923 22:11:35.146955 21078 raft_consensus.cc:2781] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:35.147094 21078 raft_consensus.cc:686] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 1 LEADER]: Becoming Leader. State: Replica: a3c90646cb1f4ec5a0c6b50fe6a0ccf2, State: Running, Role: LEADER
I20260923 22:11:35.147212 21078 consensus_queue.cc:237] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } }
I20260923 22:11:35.148139 20606 catalog_manager.cc:5495] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3c90646cb1f4ec5a0c6b50fe6a0ccf2 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3c90646cb1f4ec5a0c6b50fe6a0ccf2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 } health_report { overall_health: UNKNOWN } } }
I20260923 22:11:35.178246 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.175905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.1.244.194:42725)
0923 22:11:35.175953 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.175955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.175969 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:11:35.176075 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.176076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.176095 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.176146 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:35.176252 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.176734 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.177470 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.177656 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.177710 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.177847 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:35.177935 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:35.178006 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.178072 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.178083 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:11:35.180227 21010 raft_consensus.cc:1261] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [term 1 FOLLOWER]: Refusing update from remote peer a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:35.180267 20748 raft_consensus.cc:1261] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [term 1 FOLLOWER]: Refusing update from remote peer a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:35.180656 21080 consensus_queue.cc:1035] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a0731255c4422481174f844dbab8f5" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:35.180799 21078 consensus_queue.cc:1035] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd1c4f933934d89a2babc5693469c59" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:35.185851 20605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:35.185958 20605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:35.186789 20605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e53f19c0ac4244f7b657028b19bde9a9: Sending DeleteTablet for 3 replicas of tablet da60d80da02046ba817ec996a825f72b
I20260923 22:11:35.187137 20728 tablet_service.cc:1514] Processing DeleteTablet for tablet da60d80da02046ba817ec996a825f72b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:35 UTC) from {username='slave'} at 127.0.0.1:47598
I20260923 22:11:35.187276 20990 tablet_service.cc:1514] Processing DeleteTablet for tablet da60d80da02046ba817ec996a825f72b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:35 UTC) from {username='slave'} at 127.0.0.1:54822
I20260923 22:11:35.187259 20859 tablet_service.cc:1514] Processing DeleteTablet for tablet da60d80da02046ba817ec996a825f72b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:35 UTC) from {username='slave'} at 127.0.0.1:58904
I20260923 22:11:35.187516 21094 tablet_replica.cc:330] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: stopping tablet replica
I20260923 22:11:35.187520 21093 tablet_replica.cc:330] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5: stopping tablet replica
I20260923 22:11:35.187644 21094 raft_consensus.cc:2229] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:35.187650 21093 raft_consensus.cc:2229] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:35.187747 21094 raft_consensus.cc:2258] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:35.187749 21093 raft_consensus.cc:2258] T da60d80da02046ba817ec996a825f72b P 66a0731255c4422481174f844dbab8f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:35.188014 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 20666
I20260923 22:11:35.187894 21095 tablet_replica.cc:330] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: stopping tablet replica
I20260923 22:11:35.188105 21095 raft_consensus.cc:2229] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:11:35.188194 21095 raft_consensus.cc:2258] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:35.188220 21094 ts_tablet_manager.cc:1905] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:35.188634 21095 ts_tablet_manager.cc:1905] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:35.190238 21094 ts_tablet_manager.cc:1918] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:35.190307 21094 log.cc:1196] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/da60d80da02046ba817ec996a825f72b
I20260923 22:11:35.190294 21095 ts_tablet_manager.cc:1918] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:35.190359 21095 log.cc:1196] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/da60d80da02046ba817ec996a825f72b
I20260923 22:11:35.190582 21094 ts_tablet_manager.cc:1939] T da60d80da02046ba817ec996a825f72b P 1fd1c4f933934d89a2babc5693469c59: Deleting consensus metadata
I20260923 22:11:35.190613 21095 ts_tablet_manager.cc:1939] T da60d80da02046ba817ec996a825f72b P a3c90646cb1f4ec5a0c6b50fe6a0ccf2: Deleting consensus metadata
I20260923 22:11:35.191926 20593 catalog_manager.cc:4841] TS a3c90646cb1f4ec5a0c6b50fe6a0ccf2 (127.1.244.194:42725): tablet da60d80da02046ba817ec996a825f72b (table test-table [id=753a412f76df490ab9028f29cc972325]) successfully deleted
I20260923 22:11:35.192036 20593 catalog_manager.cc:4841] TS 1fd1c4f933934d89a2babc5693469c59 (127.1.244.195:40891): tablet da60d80da02046ba817ec996a825f72b (table test-table [id=753a412f76df490ab9028f29cc972325]) successfully deleted
W20260923 22:11:35.194429 20590 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:35413 (error 108)
I20260923 22:11:35.194664 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 20797
I20260923 22:11:35.196581 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.196443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:35413 (local address 127.0.0.1:47614)
0923 22:11:35.196489 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.196538 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:35413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260923 22:11:35.196738 20590 catalog_manager.cc:4584] TS 66a0731255c4422481174f844dbab8f5 (127.1.244.193:35413): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet da60d80da02046ba817ec996a825f72b: Network error: Client connection negotiation failed: client connection to 127.1.244.193:35413: connect: Connection refused (error 111)
I20260923 22:11:35.199743 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 20928
I20260923 22:11:35.205130 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 20575
2026-09-23T22:11:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (804 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:35 dist-test-slave-b5jz krb5kdc[21099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:35 dist-test-slave-b5jz krb5kdc[21099](info): set up 2 sockets
Sep 23 22:11:35 dist-test-slave-b5jz krb5kdc[21099](info): commencing operation
krb5kdc: starting...
W20260923 22:11:36.239616 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:36 dist-test-slave-b5jz krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:11:36.366277 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35077
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:36.448426 21112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:36.448650 21112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:36.448674 21112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:36.449786 21112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:36.449823 21112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:36.449838 21112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:36.449851 21112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:36.449864 21112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:36.451270 21112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:36.451479 21112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:36.451743 21112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:36.457033 21117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.457026 21120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.457051 21118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.457188 21112 server_base.cc:1034] running on GCE node
I20260923 22:11:36.457551 21112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:36.457916 21112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:36.459095 21112 hybrid_clock.cc:648] HybridClock initialized: now 1790201496459083 us; error 47 us; skew 500 ppm
Sep 23 22:11:36 dist-test-slave-b5jz krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:36.461851 21112 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:11:36.462312 21112 webserver.cc:460] Webserver started at http://127.1.244.254:33335/ using document root <none> and password file <none>
I20260923 22:11:36.462546 21112 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:36.462617 21112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:36.462777 21112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:36.463964 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "91b5df7f22cb48578442c75ed56b7d36"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "bfca6f5daeecb0a0b060c95873ea3b02"
server_key_iv: "32e26f07d17bf6a5bb3e65fb55cc532b"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.464335 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "91b5df7f22cb48578442c75ed56b7d36"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "bfca6f5daeecb0a0b060c95873ea3b02"
server_key_iv: "32e26f07d17bf6a5bb3e65fb55cc532b"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.465708 21112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:36.466532 21127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.466781 21112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:36.466890 21112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "91b5df7f22cb48578442c75ed56b7d36"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "bfca6f5daeecb0a0b060c95873ea3b02"
server_key_iv: "32e26f07d17bf6a5bb3e65fb55cc532b"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.466974 21112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:36.488287 21112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:36.490409 21112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:36.490597 21112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:36.494717 21112 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35077
I20260923 22:11:36.494755 21179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35077 every 8 connection(s)
I20260923 22:11:36.495127 21112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:36.496186 21180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:36.498708 21180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36: Bootstrap starting.
I20260923 22:11:36.499359 21180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:36.499727 21180 log.cc:824] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:36.500491 21180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36: No bootstrap required, opened a new log
I20260923 22:11:36.501682 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 21112
I20260923 22:11:36.501781 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:36.501987 2003 external_mini_cluster.cc:1428] Setting key 95e0457784c69a8a9a4ae37259c01128
I20260923 22:11:36.502642 21180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b5df7f22cb48578442c75ed56b7d36" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35077 } }
I20260923 22:11:36.502863 21180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:36.502908 21180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b5df7f22cb48578442c75ed56b7d36, State: Initialized, Role: FOLLOWER
I20260923 22:11:36.503043 21180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b5df7f22cb48578442c75ed56b7d36" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35077 } }
I20260923 22:11:36.503129 21180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:36.503176 21180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:36.503237 21180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:36.504591 21180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b5df7f22cb48578442c75ed56b7d36" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35077 } }
I20260923 22:11:36.504696 21180 leader_election.cc:302] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91b5df7f22cb48578442c75ed56b7d36; no voters:
I20260923 22:11:36.504907 21180 leader_election.cc:288] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:36.504963 21185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:36.505103 21185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [term 1 LEADER]: Becoming Leader. State: Replica: 91b5df7f22cb48578442c75ed56b7d36, State: Running, Role: LEADER
I20260923 22:11:36.505200 21180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:36.505203 21185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b5df7f22cb48578442c75ed56b7d36" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35077 } }
I20260923 22:11:36.505605 21187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91b5df7f22cb48578442c75ed56b7d36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b5df7f22cb48578442c75ed56b7d36" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35077 } } }
I20260923 22:11:36.505739 21187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:36.505594 21185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91b5df7f22cb48578442c75ed56b7d36. Latest consensus state: current_term: 1 leader_uuid: "91b5df7f22cb48578442c75ed56b7d36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b5df7f22cb48578442c75ed56b7d36" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35077 } } }
I20260923 22:11:36.505949 21185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:36.506142 21192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:36.506722 21192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:11:36 dist-test-slave-b5jz krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:36.511600 21192 catalog_manager.cc:1347] Generated new cluster ID: 220a1b40bb1f4924b49a26f813302873
I20260923 22:11:36.511674 21192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:36.514070 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.502613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.1.244.254:35077)
0923 22:11:36.502829 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.502840 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.502879 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:11:36.503016 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.503024 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.503086 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.503252 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:36.504340 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.505610 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.506747 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.507433 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.507542 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.510353 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:36.510357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:36.510358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:36.510371 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:36.512403 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:36.512816 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:36.512819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:36.512819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:36.512925 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:36.513149 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:36.513150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:36.513151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:36.513401 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:36.513601 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.513696 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.513843 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":102,"threads_started":1}
I20260923 22:11:36.523425 21192 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:36.524036 21192 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:36.529007 21192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36: Generated new TSK 0
I20260923 22:11:36.529223 21192 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260923 22:11:36.558693 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35077
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:36.642702 21206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:36.642927 21206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:36.643000 21206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:36.644312 21206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:36.644362 21206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:36.644392 21206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:11:36.645924 21206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:36.646188 21206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:36.646456 21206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:36.651598 21211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.651638 21212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.651639 21214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.651746 21206 server_base.cc:1034] running on GCE node
I20260923 22:11:36.651995 21206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:36.652266 21206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:36.653477 21206 hybrid_clock.cc:648] HybridClock initialized: now 1790201496653461 us; error 28 us; skew 500 ppm
Sep 23 22:11:36 dist-test-slave-b5jz krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:36.656163 21206 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260923 22:11:36.656574 21206 webserver.cc:460] Webserver started at http://127.1.244.193:45455/ using document root <none> and password file <none>
I20260923 22:11:36.656771 21206 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:36.656813 21206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:36.656904 21206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:36.657954 21206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "1ba2b50269824e928ce07562afd0201b"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "aef5ebbfa7be8b9d7020f09bdf5fa5e4"
server_key_iv: "2105d3a38ab979b83ba46e73bdd845c6"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.658282 21206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "1ba2b50269824e928ce07562afd0201b"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "aef5ebbfa7be8b9d7020f09bdf5fa5e4"
server_key_iv: "2105d3a38ab979b83ba46e73bdd845c6"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.659657 21206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:11:36.660400 21221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.660561 21206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:36.660653 21206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "1ba2b50269824e928ce07562afd0201b"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "aef5ebbfa7be8b9d7020f09bdf5fa5e4"
server_key_iv: "2105d3a38ab979b83ba46e73bdd845c6"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.660708 21206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:36.668365 21206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:36.670688 21206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:36.670858 21206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:36.671063 21206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:36.671305 21206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:36.671335 21206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.671357 21206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:36.671370 21206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.677022 21206 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:36277
I20260923 22:11:36.677142 21334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:36277 every 8 connection(s)
I20260923 22:11:36.677376 21206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:11:36 dist-test-slave-b5jz krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:36.683689 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 21206
I20260923 22:11:36.683806 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:36.684029 2003 external_mini_cluster.cc:1428] Setting key 84dfc1958d94a1b75a0adab1f5758fce
I20260923 22:11:36.687235 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.678134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:35281 (local address 127.1.244.254:35077)
0923 22:11:36.678199 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.678201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.678415 (+ 214us) server_negotiation.cc:407] Connection header received
0923 22:11:36.678905 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.678906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.678919 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.678987 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:36.679966 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.680473 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.681476 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.681676 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.681734 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.684015 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:36.684016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:36.684016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:36.684023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:36.685666 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:36.686219 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:36.686220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:36.686220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:36.686292 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:36.686543 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:36.686543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:36.686544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:36.686703 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:36.686775 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.687034 (+ 259us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.687134 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:11:36.687489 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.678105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35077 (local address 127.1.244.193:35281)
0923 22:11:36.678284 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.678327 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.678818 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.679062 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.679066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.679686 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:36.679878 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.679883 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.680554 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.680555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.681379 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.681381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.681757 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.681758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.681860 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.682349 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:36.682358 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:36.683897 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:36.685752 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:36.685760 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:36.685762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:36.686114 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:36.686368 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:36.686368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:36.686368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:36.686472 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:36.686770 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:36.686774 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:36.686892 (+ 118us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.686997 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.687128 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":109,"threads_started":1}
I20260923 22:11:36.687965 21335 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35077
I20260923 22:11:36.688081 21335 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:36.688261 21335 heartbeater.cc:511] Master 127.1.244.254:35077 requested a full tablet report, sending...
I20260923 22:11:36.688793 21144 ts_manager.cc:194] Registered new tserver with Master: 1ba2b50269824e928ce07562afd0201b (127.1.244.193:36277)
I20260923 22:11:36.689426 21144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:35281
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260923 22:11:36.710611 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35077
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:36.796530 21340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:36.796739 21340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:36.796764 21340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:36.797857 21340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:36.797892 21340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:36.797917 21340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:11:36.799297 21340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:36.799497 21340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:36.799748 21340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:36.804836 21345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.804844 21346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.804853 21348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.805039 21340 server_base.cc:1034] running on GCE node
I20260923 22:11:36.805356 21340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:36.805609 21340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:36.806752 21340 hybrid_clock.cc:648] HybridClock initialized: now 1790201496806743 us; error 28 us; skew 500 ppm
Sep 23 22:11:36 dist-test-slave-b5jz krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:36.809341 21340 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260923 22:11:36.809808 21340 webserver.cc:460] Webserver started at http://127.1.244.194:43425/ using document root <none> and password file <none>
I20260923 22:11:36.810022 21340 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:36.810063 21340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:36.810153 21340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:36.811152 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "bb04bc498c2a42e78b5e6fac79d6316e"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "b327e9bcba41145fc1b4677d9fe46eb9"
server_key_iv: "4ad6b462c0a88cd5047316ab1af15e44"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.811448 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "bb04bc498c2a42e78b5e6fac79d6316e"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "b327e9bcba41145fc1b4677d9fe46eb9"
server_key_iv: "4ad6b462c0a88cd5047316ab1af15e44"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.812819 21340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:36.813598 21355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.813822 21340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:36.813903 21340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "bb04bc498c2a42e78b5e6fac79d6316e"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "b327e9bcba41145fc1b4677d9fe46eb9"
server_key_iv: "4ad6b462c0a88cd5047316ab1af15e44"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.813962 21340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:36.835145 21340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:36.837525 21340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:36.837677 21340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:36.837965 21340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:36.838306 21340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:36.838342 21340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.838366 21340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:36.838418 21340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.844259 21340 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:34311
I20260923 22:11:36.844338 21468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:34311 every 8 connection(s)
I20260923 22:11:36.844673 21340 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:36.845717 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 21340
I20260923 22:11:36.845836 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:36.846031 2003 external_mini_cluster.cc:1428] Setting key 990dc396906b3e75eb9e4d57b5ce4493
Sep 23 22:11:36 dist-test-slave-b5jz krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
I20260923 22:11:36.856138 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.845446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:53997 (local address 127.1.244.254:35077)
0923 22:11:36.845502 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.845504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.845714 (+ 210us) server_negotiation.cc:407] Connection header received
0923 22:11:36.846341 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.846343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.846356 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.846433 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:36.847501 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.848323 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.849818 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.850165 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.850245 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.852479 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:36.852480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:36.852481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:36.852489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:36.854153 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:36.854694 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:36.854695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:36.854696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:36.854797 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:36.855182 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:36.855183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:36.855184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:36.855435 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:36.855513 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.855951 (+ 438us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.856043 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:11:36.856616 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.845396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35077 (local address 127.1.244.194:53997)
0923 22:11:36.845561 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.845604 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.846218 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.846508 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.846511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.847176 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:36.847386 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.847391 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.848432 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.848433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.849673 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.849676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.850253 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.850254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.850361 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.850949 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:36.850958 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:36.852375 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:36.854247 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:36.854256 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:36.854259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:36.854579 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:36.854920 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:36.854921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:36.854921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:36.855073 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:36.855566 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:36.855574 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:36.855722 (+ 148us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.855915 (+ 193us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.856095 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260923 22:11:36.857307 21469 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35077
I20260923 22:11:36.857409 21469 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:36.857563 21469 heartbeater.cc:511] Master 127.1.244.254:35077 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
I20260923 22:11:36.858017 21144 ts_manager.cc:194] Registered new tserver with Master: bb04bc498c2a42e78b5e6fac79d6316e (127.1.244.194:34311)
I20260923 22:11:36.858525 21144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:53997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260923 22:11:36.875078 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35077
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:36.958554 21474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:36.958819 21474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:36.958859 21474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:36.960198 21474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:36.960268 21474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:36.960311 21474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:11:36.961800 21474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:36.962077 21474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:36.962314 21474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:36.967499 21482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.967618 21480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.967509 21479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.967885 21474 server_base.cc:1034] running on GCE node
I20260923 22:11:36.968068 21474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:36.968278 21474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:36.969437 21474 hybrid_clock.cc:648] HybridClock initialized: now 1790201496969423 us; error 28 us; skew 500 ppm
Sep 23 22:11:36 dist-test-slave-b5jz krb5kdc[21099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:36.972121 21474 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260923 22:11:36.972563 21474 webserver.cc:460] Webserver started at http://127.1.244.195:46613/ using document root <none> and password file <none>
I20260923 22:11:36.972833 21474 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:36.972908 21474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:36.973052 21474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:36.974291 21474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "e6185b9c249247668c6b3b8a2de63af0"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "f313c815af419cc91b187b597b0da344"
server_key_iv: "c889029d2fff41d086eb6f46a3660a45"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.974666 21474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "e6185b9c249247668c6b3b8a2de63af0"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "f313c815af419cc91b187b597b0da344"
server_key_iv: "c889029d2fff41d086eb6f46a3660a45"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.976101 21474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:36.976943 21489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.977157 21474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:36.977262 21474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "e6185b9c249247668c6b3b8a2de63af0"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-b5jz"
server_key: "f313c815af419cc91b187b597b0da344"
server_key_iv: "c889029d2fff41d086eb6f46a3660a45"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.977345 21474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:36.991151 21474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:36.993654 21474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:36.993832 21474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:36.994077 21474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:36.994405 21474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:36.994470 21474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.994519 21474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:36.994547 21474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.000553 21474 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:41187
I20260923 22:11:37.000630 21602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:41187 every 8 connection(s)
I20260923 22:11:37.000963 21474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:11:37 dist-test-slave-b5jz krb5kdc[21099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:37.010224 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 21474
I20260923 22:11:37.010314 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:37.010545 2003 external_mini_cluster.cc:1428] Setting key d939e23f856bb6e3313251735127896e
I20260923 22:11:37.010960 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.001755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:47725 (local address 127.1.244.254:35077)
0923 22:11:37.001823 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.001825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.002053 (+ 228us) server_negotiation.cc:407] Connection header received
0923 22:11:37.002612 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.002613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.002626 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.002674 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:37.003725 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.004234 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.005346 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.005661 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.005736 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.008145 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:37.008146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:37.008147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:37.008153 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:37.009439 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:37.009837 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:37.009838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:37.009838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:37.009887 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:37.010172 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:37.010173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:37.010174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:37.010348 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:37.010414 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.010736 (+ 322us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.010844 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:11:37.011179 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.001724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35077 (local address 127.1.244.195:47725)
0923 22:11:37.001921 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.001963 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.002492 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.002744 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.002749 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.003361 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:37.003619 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.003626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.004332 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.004334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.005240 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.005243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.005752 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.005753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.005880 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.006453 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:37.006461 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:37.008055 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:37.009530 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:37.009537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:37.009539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:37.009768 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:37.009945 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:37.009946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:37.009946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:37.010069 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:37.010442 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:37.010453 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:37.010593 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.010707 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.010906 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260923 22:11:37.011659 21603 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35077
I20260923 22:11:37.011770 21603 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:37.011958 21603 heartbeater.cc:511] Master 127.1.244.254:35077 requested a full tablet report, sending...
I20260923 22:11:37.012360 21144 ts_manager.cc:194] Registered new tserver with Master: e6185b9c249247668c6b3b8a2de63af0 (127.1.244.195:41187)
I20260923 22:11:37.012702 21144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:47725
I20260923 22:11:37.022550 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:37.030062 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.023861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.1.244.254:35077)
0923 22:11:37.023949 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.023951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.024067 (+ 116us) server_negotiation.cc:407] Connection header received
0923 22:11:37.024095 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.024096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.024106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.024155 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:37.024714 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.025209 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.025908 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.026098 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.026166 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.027369 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:37.027370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:37.027370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:37.027375 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:37.028742 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:37.029143 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:37.029145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:37.029145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:37.029205 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:37.029416 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:37.029416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:37.029417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:37.029666 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:37.029779 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.029881 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.029956 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:11:37.031841 21144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:37.033519 21144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:37.040295 21617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.036108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.1.244.194:34311)
0923 22:11:37.036491 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.036497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.036508 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:11:37.036551 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.036553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.036571 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.036645 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:37.036827 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.037988 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.039133 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.039817 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.039915 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.039975 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.040153 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.040199 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":72,"threads_started":1}
I20260923 22:11:37.040323 21614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.035903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:34311 (local address 127.0.0.1:32798)
0923 22:11:37.036087 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.036130 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.036194 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.036664 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.036668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.036670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:37.036743 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.036745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.038105 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.038106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.039023 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.039025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.039918 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.039919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.040015 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.040028 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.040134 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.040211 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":29,"thread_start_us":71,"threads_started":1}
I20260923 22:11:37.040396 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.035957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41187 (local address 127.0.0.1:49276)
0923 22:11:37.036265 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.036275 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.036332 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.036960 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.036961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.036963 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:37.037003 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.037004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.038219 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.038220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.039285 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.039287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.040163 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.040164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.040217 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.040220 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.040251 (+ 31us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.040289 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":77,"threads_started":1}
I20260923 22:11:37.040692 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.036357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46636 (local address 127.1.244.193:36277)
0923 22:11:37.036519 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.036524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.036538 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:11:37.036596 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.036598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.036615 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.036692 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:37.036934 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.037678 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.039089 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.039838 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.039911 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.039969 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.040332 (+ 363us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.040609 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260923 22:11:37.040781 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.036073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49276 (local address 127.1.244.195:41187)
0923 22:11:37.036787 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.036791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.036801 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:11:37.036841 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.036843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.036859 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.036963 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:37.037078 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.038117 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.039369 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.040071 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.040563 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.040626 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.040651 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.040692 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":93,"threads_started":1}
I20260923 22:11:37.041049 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.036118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36277 (local address 127.0.0.1:46636)
0923 22:11:37.036337 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.036352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.036417 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.036810 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.036811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.036812 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:37.036858 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.036859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.037784 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.037785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.038994 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.038996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.040005 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.040006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.040069 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.040074 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.040127 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.040207 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":81,"threads_started":1}
I20260923 22:11:37.041332 21537 tablet_service.cc:1467] Processing CreateTablet for tablet 09d470e04a964c9babfa3df77010b005 (DEFAULT_TABLE table=test-table [id=96de62f506354f1e8eefafb8125944da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:37.041301 21403 tablet_service.cc:1467] Processing CreateTablet for tablet 09d470e04a964c9babfa3df77010b005 (DEFAULT_TABLE table=test-table [id=96de62f506354f1e8eefafb8125944da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:37.041671 21403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09d470e04a964c9babfa3df77010b005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:37.041671 21537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09d470e04a964c9babfa3df77010b005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:37.041719 21269 tablet_service.cc:1467] Processing CreateTablet for tablet 09d470e04a964c9babfa3df77010b005 (DEFAULT_TABLE table=test-table [id=96de62f506354f1e8eefafb8125944da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:37.041965 21269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09d470e04a964c9babfa3df77010b005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:37.044703 21620 tablet_bootstrap.cc:492] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b: Bootstrap starting.
I20260923 22:11:37.045056 21621 tablet_bootstrap.cc:492] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: Bootstrap starting.
I20260923 22:11:37.045426 21620 tablet_bootstrap.cc:654] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:37.045634 21621 tablet_bootstrap.cc:654] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:37.045994 21620 log.cc:824] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:37.046134 21621 log.cc:824] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:37.046355 21622 tablet_bootstrap.cc:492] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: Bootstrap starting.
I20260923 22:11:37.046779 21621 tablet_bootstrap.cc:492] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: No bootstrap required, opened a new log
I20260923 22:11:37.046789 21620 tablet_bootstrap.cc:492] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b: No bootstrap required, opened a new log
I20260923 22:11:37.046864 21620 ts_tablet_manager.cc:1397] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:11:37.046865 21621 ts_tablet_manager.cc:1397] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:37.047068 21622 tablet_bootstrap.cc:654] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:37.047711 21622 log.cc:824] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:37.048538 21622 tablet_bootstrap.cc:492] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: No bootstrap required, opened a new log
I20260923 22:11:37.048605 21622 ts_tablet_manager.cc:1397] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:11:37.048743 21621 raft_consensus.cc:348] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } }
I20260923 22:11:37.048903 21621 raft_consensus.cc:374] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:37.048946 21621 raft_consensus.cc:729] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb04bc498c2a42e78b5e6fac79d6316e, State: Initialized, Role: FOLLOWER
I20260923 22:11:37.049081 21621 consensus_queue.cc:260] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } }
I20260923 22:11:37.049078 21620 raft_consensus.cc:348] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } }
I20260923 22:11:37.049248 21620 raft_consensus.cc:374] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:37.049295 21620 raft_consensus.cc:729] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ba2b50269824e928ce07562afd0201b, State: Initialized, Role: FOLLOWER
I20260923 22:11:37.049358 21621 ts_tablet_manager.cc:1428] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:11:37.049432 21620 consensus_queue.cc:260] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } }
I20260923 22:11:37.049499 21469 heartbeater.cc:503] Master 127.1.244.254:35077 was elected leader, sending a full tablet report...
I20260923 22:11:37.049734 21335 heartbeater.cc:503] Master 127.1.244.254:35077 was elected leader, sending a full tablet report...
I20260923 22:11:37.049714 21620 ts_tablet_manager.cc:1428] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:37.050639 21622 raft_consensus.cc:348] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } }
I20260923 22:11:37.050827 21622 raft_consensus.cc:374] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:37.050877 21622 raft_consensus.cc:729] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6185b9c249247668c6b3b8a2de63af0, State: Initialized, Role: FOLLOWER
I20260923 22:11:37.050997 21622 consensus_queue.cc:260] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } }
I20260923 22:11:37.051316 21622 ts_tablet_manager.cc:1428] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:11:37.051318 21603 heartbeater.cc:503] Master 127.1.244.254:35077 was elected leader, sending a full tablet report...
W20260923 22:11:37.095240 21470 tablet.cc:2369] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:37.157624 21626 raft_consensus.cc:482] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:37.157764 21626 raft_consensus.cc:504] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } }
I20260923 22:11:37.158234 21626 leader_election.cc:288] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ba2b50269824e928ce07562afd0201b (127.1.244.193:36277), e6185b9c249247668c6b3b8a2de63af0 (127.1.244.195:41187)
I20260923 22:11:37.162189 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.158501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:53119 (local address 127.1.244.195:41187)
0923 22:11:37.158546 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.158548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.158559 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:11:37.158613 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.158614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.158624 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.158677 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:37.158890 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.159697 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.161235 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.161819 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.161888 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.161896 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.162077 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.162118 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:11:37.162189 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.158550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:48001 (local address 127.1.244.193:36277)
0923 22:11:37.158592 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.158594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.158730 (+ 136us) server_negotiation.cc:407] Connection header received
0923 22:11:37.158773 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.158774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.158782 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.158819 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:37.158984 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.159697 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.161235 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.161781 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.161877 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.161886 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.162077 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.162118 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:11:37.162206 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.158414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41187 (local address 127.1.244.194:53119)
0923 22:11:37.158473 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.158487 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.158545 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.158711 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.158712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.158714 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:37.158789 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.158791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.159781 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.159782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.161126 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.161129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.161899 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.161900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.161995 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.161996 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.162058 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.162109 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":63}
I20260923 22:11:37.162213 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.158492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36277 (local address 127.1.244.194:48001)
0923 22:11:37.158652 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.158660 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.158711 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.158845 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.158846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.158848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:37.158909 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.158914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.159781 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.159782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.161126 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.161129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.161899 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.161900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.161990 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.161995 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.162058 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.162108 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":85,"thread_start_us":100,"threads_started":1}
I20260923 22:11:37.162591 21289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09d470e04a964c9babfa3df77010b005" candidate_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba2b50269824e928ce07562afd0201b" is_pre_election: true
I20260923 22:11:37.162573 21557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09d470e04a964c9babfa3df77010b005" candidate_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6185b9c249247668c6b3b8a2de63af0" is_pre_election: true
I20260923 22:11:37.162755 21289 raft_consensus.cc:2454] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb04bc498c2a42e78b5e6fac79d6316e in term 0.
I20260923 22:11:37.162755 21557 raft_consensus.cc:2454] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb04bc498c2a42e78b5e6fac79d6316e in term 0.
I20260923 22:11:37.162974 21356 leader_election.cc:302] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb04bc498c2a42e78b5e6fac79d6316e, e6185b9c249247668c6b3b8a2de63af0; no voters:
I20260923 22:11:37.163098 21626 raft_consensus.cc:2781] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:37.163153 21626 raft_consensus.cc:482] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:37.163188 21626 raft_consensus.cc:3037] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:37.164146 21626 raft_consensus.cc:504] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } }
I20260923 22:11:37.164294 21626 leader_election.cc:288] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [CANDIDATE]: Term 1 election: Requested vote from peers 1ba2b50269824e928ce07562afd0201b (127.1.244.193:36277), e6185b9c249247668c6b3b8a2de63af0 (127.1.244.195:41187)
I20260923 22:11:37.164439 21289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09d470e04a964c9babfa3df77010b005" candidate_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba2b50269824e928ce07562afd0201b"
I20260923 22:11:37.164470 21557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09d470e04a964c9babfa3df77010b005" candidate_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6185b9c249247668c6b3b8a2de63af0"
I20260923 22:11:37.164513 21289 raft_consensus.cc:3037] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:37.164588 21557 raft_consensus.cc:3037] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:37.165251 21289 raft_consensus.cc:2454] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb04bc498c2a42e78b5e6fac79d6316e in term 1.
I20260923 22:11:37.165398 21557 raft_consensus.cc:2454] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb04bc498c2a42e78b5e6fac79d6316e in term 1.
I20260923 22:11:37.165423 21359 leader_election.cc:302] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ba2b50269824e928ce07562afd0201b, bb04bc498c2a42e78b5e6fac79d6316e; no voters:
I20260923 22:11:37.165518 21626 raft_consensus.cc:2781] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:37.165684 21626 raft_consensus.cc:686] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 1 LEADER]: Becoming Leader. State: Replica: bb04bc498c2a42e78b5e6fac79d6316e, State: Running, Role: LEADER
I20260923 22:11:37.165792 21626 consensus_queue.cc:237] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } }
I20260923 22:11:37.166581 21144 catalog_manager.cc:5495] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb04bc498c2a42e78b5e6fac79d6316e (127.1.244.194). New cstate: current_term: 1 leader_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb04bc498c2a42e78b5e6fac79d6316e" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 34311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 } health_report { overall_health: UNKNOWN } } }
W20260923 22:11:37.178110 21336 tablet.cc:2369] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:37.184218 21617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.181906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32814 (local address 127.1.244.194:34311)
0923 22:11:37.181949 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.181951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.181959 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:11:37.182026 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.182027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.182043 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.182102 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:37.182215 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.182685 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.183393 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.183638 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.183697 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.183892 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:37.183999 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:37.184063 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.184095 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.184112 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:11:37.186141 21289 raft_consensus.cc:1261] T 09d470e04a964c9babfa3df77010b005 P 1ba2b50269824e928ce07562afd0201b [term 1 FOLLOWER]: Refusing update from remote peer bb04bc498c2a42e78b5e6fac79d6316e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:37.186154 21557 raft_consensus.cc:1261] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [term 1 FOLLOWER]: Refusing update from remote peer bb04bc498c2a42e78b5e6fac79d6316e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:37.186498 21626 consensus_queue.cc:1035] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ba2b50269824e928ce07562afd0201b" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:37.186645 21630 consensus_queue.cc:1035] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6185b9c249247668c6b3b8a2de63af0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:37.188931 21633 mvcc.cc:205] Tried to move back new op lower bound from 7332665332471599104 to 7332665332391714816. Current Snapshot: MvccSnapshot[applied={T|T < 7332665332471599104}]
I20260923 22:11:37.189172 21634 mvcc.cc:205] Tried to move back new op lower bound from 7332665332471599104 to 7332665332391714816. Current Snapshot: MvccSnapshot[applied={T|T < 7332665332471599104}]
I20260923 22:11:37.189800 21635 mvcc.cc:205] Tried to move back new op lower bound from 7332665332471599104 to 7332665332391714816. Current Snapshot: MvccSnapshot[applied={T|T < 7332665332471599104}]
I20260923 22:11:37.191166 21143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:37.191264 21143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:37.192072 21143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91b5df7f22cb48578442c75ed56b7d36: Sending DeleteTablet for 3 replicas of tablet 09d470e04a964c9babfa3df77010b005
I20260923 22:11:37.192442 21403 tablet_service.cc:1514] Processing DeleteTablet for tablet 09d470e04a964c9babfa3df77010b005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:37 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:32798
I20260923 22:11:37.192456 21269 tablet_service.cc:1514] Processing DeleteTablet for tablet 09d470e04a964c9babfa3df77010b005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:37 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46636
I20260923 22:11:37.192481 21537 tablet_service.cc:1514] Processing DeleteTablet for tablet 09d470e04a964c9babfa3df77010b005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:37 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:49276
I20260923 22:11:37.192675 21644 tablet_replica.cc:330] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: stopping tablet replica
I20260923 22:11:37.192791 21644 raft_consensus.cc:2229] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:37.192896 21644 raft_consensus.cc:2258] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:37.192975 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 21206
I20260923 22:11:37.193346 21644 ts_tablet_manager.cc:1905] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:37.195359 21644 ts_tablet_manager.cc:1918] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:37.195439 21644 log.cc:1196] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/09d470e04a964c9babfa3df77010b005
I20260923 22:11:37.195739 21644 ts_tablet_manager.cc:1939] T 09d470e04a964c9babfa3df77010b005 P e6185b9c249247668c6b3b8a2de63af0: Deleting consensus metadata
I20260923 22:11:37.193081 21643 tablet_replica.cc:330] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: stopping tablet replica
I20260923 22:11:37.195951 21643 raft_consensus.cc:2229] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:11:37.196111 21643 raft_consensus.cc:2258] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:37.196511 21643 ts_tablet_manager.cc:1905] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:37.196619 21128 catalog_manager.cc:4841] TS e6185b9c249247668c6b3b8a2de63af0 (127.1.244.195:41187): tablet 09d470e04a964c9babfa3df77010b005 (table test-table [id=96de62f506354f1e8eefafb8125944da]) successfully deleted
I20260923 22:11:37.198362 21643 ts_tablet_manager.cc:1918] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:37.198439 21643 log.cc:1196] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/09d470e04a964c9babfa3df77010b005
I20260923 22:11:37.198691 21643 ts_tablet_manager.cc:1939] T 09d470e04a964c9babfa3df77010b005 P bb04bc498c2a42e78b5e6fac79d6316e: Deleting consensus metadata
I20260923 22:11:37.199103 21131 catalog_manager.cc:4841] TS bb04bc498c2a42e78b5e6fac79d6316e (127.1.244.194:34311): tablet 09d470e04a964c9babfa3df77010b005 (table test-table [id=96de62f506354f1e8eefafb8125944da]) successfully deleted
W20260923 22:11:37.200992 21131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:36277 (error 108)
I20260923 22:11:37.201339 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 21340
I20260923 22:11:37.201506 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.201390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36277 (local address 127.0.0.1:46648)
0923 22:11:37.201441 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.201475 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 22:11:37.202056 21131 catalog_manager.cc:4584] TS 1ba2b50269824e928ce07562afd0201b (127.1.244.193:36277): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09d470e04a964c9babfa3df77010b005: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36277: connect: Connection refused (error 111)
I20260923 22:11:37.205991 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 21474
I20260923 22:11:37.210848 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 21112
2026-09-23T22:11:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2006 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-23T22:11:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:37Z Disabled control of system clock
I20260923 22:11:37.238600 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38441
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:37.318863 21649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:37.319139 21649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:37.319176 21649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:37.320393 21649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:37.320474 21649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:37.320497 21649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:37.320518 21649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:37.320550 21649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:37.321911 21649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38441
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:37.322206 21649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:37.322515 21649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:37.327399 21655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:37.327399 21654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:37.327416 21657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:37.327838 21649 server_base.cc:1034] running on GCE node
I20260923 22:11:37.328089 21649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:37.328398 21649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:37.329566 21649 hybrid_clock.cc:648] HybridClock initialized: now 1790201497329566 us; error 40 us; skew 500 ppm
I20260923 22:11:37.330828 21649 webserver.cc:460] Webserver started at http://127.1.244.254:41795/ using document root <none> and password file <none>
I20260923 22:11:37.331102 21649 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:37.331176 21649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:37.331341 21649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:37.332516 21649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "202c089baf8349ad8ddbfdc269bcc4f0"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "8a240bc9c5b493cabb5fc72b50ea54cb"
server_key_iv: "570e5a598d3da087434a156e49299798"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.332912 21649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "202c089baf8349ad8ddbfdc269bcc4f0"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "8a240bc9c5b493cabb5fc72b50ea54cb"
server_key_iv: "570e5a598d3da087434a156e49299798"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.334208 21649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:11:37.334940 21663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.335134 21649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:37.335234 21649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "202c089baf8349ad8ddbfdc269bcc4f0"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "8a240bc9c5b493cabb5fc72b50ea54cb"
server_key_iv: "570e5a598d3da087434a156e49299798"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.335322 21649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:37.362608 21649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:37.362991 21649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:37.363170 21649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:37.367365 21649 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:38441
I20260923 22:11:37.367408 21715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:38441 every 8 connection(s)
I20260923 22:11:37.367830 21649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:37.368856 21716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:37.371186 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0: Bootstrap starting.
I20260923 22:11:37.371810 21716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:37.372124 21716 log.cc:824] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:37.372812 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0: No bootstrap required, opened a new log
I20260923 22:11:37.373253 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 21649
I20260923 22:11:37.373344 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:37.373550 2003 external_mini_cluster.cc:1428] Setting key a00e21e3ef9eb9e09175ed017ac07ee1
I20260923 22:11:37.374948 21716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "202c089baf8349ad8ddbfdc269bcc4f0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38441 } }
I20260923 22:11:37.375123 21716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:37.375160 21716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 202c089baf8349ad8ddbfdc269bcc4f0, State: Initialized, Role: FOLLOWER
I20260923 22:11:37.375293 21716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "202c089baf8349ad8ddbfdc269bcc4f0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38441 } }
I20260923 22:11:37.375389 21716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:37.375427 21716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:37.375473 21716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:37.376046 21716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "202c089baf8349ad8ddbfdc269bcc4f0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38441 } }
I20260923 22:11:37.376134 21716 leader_election.cc:302] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 202c089baf8349ad8ddbfdc269bcc4f0; no voters:
I20260923 22:11:37.376320 21716 leader_election.cc:288] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:37.376355 21721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:37.376500 21721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [term 1 LEADER]: Becoming Leader. State: Replica: 202c089baf8349ad8ddbfdc269bcc4f0, State: Running, Role: LEADER
I20260923 22:11:37.376601 21716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:37.376586 21721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "202c089baf8349ad8ddbfdc269bcc4f0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38441 } }
I20260923 22:11:37.376902 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 202c089baf8349ad8ddbfdc269bcc4f0. Latest consensus state: current_term: 1 leader_uuid: "202c089baf8349ad8ddbfdc269bcc4f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "202c089baf8349ad8ddbfdc269bcc4f0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38441 } } }
I20260923 22:11:37.376967 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:37.376910 21722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "202c089baf8349ad8ddbfdc269bcc4f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "202c089baf8349ad8ddbfdc269bcc4f0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38441 } } }
I20260923 22:11:37.377007 21722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0 [sys.catalog]: This master's current role is: LEADER
W20260923 22:11:37.378461 21737 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:11:37.378521 21737 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:11:37.378571 21726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:37.378755 21726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:37.380283 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.374192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56104 (local address 127.1.244.254:38441)
0923 22:11:37.374382 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.374391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.374416 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:11:37.374926 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.374932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.374989 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.375096 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:37.375354 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.377110 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.378185 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.378680 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.378777 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.379496 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:37.379499 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:37.379500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:37.379728 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:37.379773 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:37.379812 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:37.379900 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.379997 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.380085 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260923 22:11:37.380756 21726 catalog_manager.cc:1347] Generated new cluster ID: b54cce5db746417980a94dd71d4e7a4e
I20260923 22:11:37.380810 21726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:37.405900 21726 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:37.406461 21726 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:37.410524 21726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0: Generated new TSK 0
I20260923 22:11:37.410737 21726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:37.417661 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38441
--builtin_ntp_servers=127.1.244.212:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:37.509522 21740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:37.509840 21740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:37.509878 21740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:37.511003 21740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:37.511070 21740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:37.511113 21740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260923 22:11:37.512524 21740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:37.512827 21740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:37.513166 21740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:37.518812 21746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:37.518826 21745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:37.518929 21748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:37.519233 21740 server_base.cc:1034] running on GCE node
I20260923 22:11:37.519445 21740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:37.519749 21740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:37.520910 21740 hybrid_clock.cc:648] HybridClock initialized: now 1790201497520893 us; error 32 us; skew 500 ppm
I20260923 22:11:37.522271 21740 webserver.cc:460] Webserver started at http://127.1.244.193:36445/ using document root <none> and password file <none>
I20260923 22:11:37.522534 21740 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:37.522624 21740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:37.522778 21740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:37.524006 21740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/data/instance:
uuid: "5c018817db5e4842a5029d7fcdc3d732"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "04e72ebf07ba9fbd878ccdcc1b847f10"
server_key_iv: "80fadc8d1a5b952950c63b296d043fa4"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.524374 21740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "5c018817db5e4842a5029d7fcdc3d732"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "04e72ebf07ba9fbd878ccdcc1b847f10"
server_key_iv: "80fadc8d1a5b952950c63b296d043fa4"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.525831 21740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:37.526619 21754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.526772 21740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:37.526875 21740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/wal
uuid: "5c018817db5e4842a5029d7fcdc3d732"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "04e72ebf07ba9fbd878ccdcc1b847f10"
server_key_iv: "80fadc8d1a5b952950c63b296d043fa4"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.526947 21740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:37.549203 21740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:37.549557 21740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:37.549722 21740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:37.549973 21740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:37.550328 21740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:37.550369 21740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.550413 21740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:37.550443 21740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.556454 21740 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:40153
I20260923 22:11:37.556505 21867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:40153 every 8 connection(s)
I20260923 22:11:37.556828 21740 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:37.562206 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.557549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:56611 (local address 127.1.244.254:38441)
0923 22:11:37.557600 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.557602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.557865 (+ 263us) server_negotiation.cc:407] Connection header received
0923 22:11:37.558428 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.558429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.558447 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.558493 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:37.559208 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.559750 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.560772 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.560978 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.561061 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.561905 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:37.561906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:37.561907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:37.561912 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:37.561934 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:37.561938 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:37.561997 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.562093 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.562125 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:11:37.562423 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.557540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38441 (local address 127.1.244.193:56611)
0923 22:11:37.557728 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.557763 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.558302 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.558577 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.558581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.558591 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:37.559099 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.559103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.559853 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.559854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.560653 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.560656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.561068 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.561069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.561210 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.561721 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:37.561721 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:37.561750 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:37.561773 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:37.561999 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:37.562006 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:37.562009 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.562098 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.562201 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260923 22:11:37.562963 21868 heartbeater.cc:347] Connected to a master server at 127.1.244.254:38441
I20260923 22:11:37.563073 21868 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:37.563220 21868 heartbeater.cc:511] Master 127.1.244.254:38441 requested a full tablet report, sending...
I20260923 22:11:37.563660 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 21740
I20260923 22:11:37.563771 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:37.563812 21680 ts_manager.cc:194] Registered new tserver with Master: 5c018817db5e4842a5029d7fcdc3d732 (127.1.244.193:40153)
I20260923 22:11:37.563977 2003 external_mini_cluster.cc:1428] Setting key 2ecd04952d90b597ada6e7e631ae553a
I20260923 22:11:37.564630 21680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.193:56611
I20260923 22:11:37.565148 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38441
--builtin_ntp_servers=127.1.244.212:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:37.646862 21871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:37.647078 21871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:37.647097 21871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:37.648178 21871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:37.648213 21871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:37.648237 21871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260923 22:11:37.649505 21871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:37.649699 21871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:37.649940 21871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:37.654929 21876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:37.654922 21877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:37.654919 21879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:37.654958 21871 server_base.cc:1034] running on GCE node
I20260923 22:11:37.655347 21871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:37.655668 21871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:37.656862 21871 hybrid_clock.cc:648] HybridClock initialized: now 1790201497656847 us; error 45 us; skew 500 ppm
I20260923 22:11:37.658092 21871 webserver.cc:460] Webserver started at http://127.1.244.194:37991/ using document root <none> and password file <none>
I20260923 22:11:37.658363 21871 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:37.658435 21871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:37.658595 21871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:37.659749 21871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/data/instance:
uuid: "776b2fcf415a4ad5a09b3d8413856a85"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "e3e5d0c9a77cf72402b5ad80dfad1116"
server_key_iv: "823298a7b8ad87eff984d426cf671250"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.660108 21871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "776b2fcf415a4ad5a09b3d8413856a85"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "e3e5d0c9a77cf72402b5ad80dfad1116"
server_key_iv: "823298a7b8ad87eff984d426cf671250"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.661396 21871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:37.662165 21885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.662349 21871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:37.662452 21871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/wal
uuid: "776b2fcf415a4ad5a09b3d8413856a85"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "e3e5d0c9a77cf72402b5ad80dfad1116"
server_key_iv: "823298a7b8ad87eff984d426cf671250"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.662533 21871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:37.671934 21871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:37.672225 21871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:37.672389 21871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:37.672609 21871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:37.672964 21871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:37.673027 21871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.673069 21871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:37.673097 21871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.679171 21871 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:35787
I20260923 22:11:37.679224 21998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:35787 every 8 connection(s)
I20260923 22:11:37.679610 21871 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:37.679788 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 21871
I20260923 22:11:37.679903 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:37.680147 2003 external_mini_cluster.cc:1428] Setting key c9cffae38d56dd0e289f87aaf5873b3c
I20260923 22:11:37.681170 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38441
--builtin_ntp_servers=127.1.244.212:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:11:37.685664 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.680698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:57109 (local address 127.1.244.254:38441)
0923 22:11:37.680744 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.680746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.680964 (+ 218us) server_negotiation.cc:407] Connection header received
0923 22:11:37.681621 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.681622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.681634 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.681686 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:37.682486 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.682992 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.684112 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.684389 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.684481 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.685367 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:37.685369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:37.685369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:37.685374 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:37.685390 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:37.685394 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:37.685445 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.685550 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.685582 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:11:37.685904 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.680389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38441 (local address 127.1.244.194:57109)
0923 22:11:37.680836 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.680886 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.681504 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.681771 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.681775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.681784 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:37.682382 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.682387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.683089 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.683090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.684016 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.684018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.684499 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.684500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.684634 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.685214 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:37.685215 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:37.685244 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:37.685267 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:37.685444 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:37.685450 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:37.685453 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.685551 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.685665 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":98,"threads_started":1}
I20260923 22:11:37.686322 21999 heartbeater.cc:347] Connected to a master server at 127.1.244.254:38441
I20260923 22:11:37.686434 21999 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:37.686647 21999 heartbeater.cc:511] Master 127.1.244.254:38441 requested a full tablet report, sending...
I20260923 22:11:37.687021 21680 ts_manager.cc:194] Registered new tserver with Master: 776b2fcf415a4ad5a09b3d8413856a85 (127.1.244.194:35787)
I20260923 22:11:37.687395 21680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.194:57109
W20260923 22:11:37.762123 22002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:37.762323 22002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:37.762344 22002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:37.763494 22002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:37.763536 22002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:37.763588 22002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260923 22:11:37.765026 22002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:38441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:37.765276 22002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:37.765568 22002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:37.770466 22007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:37.770534 22008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:37.770707 22002 server_base.cc:1034] running on GCE node
W20260923 22:11:37.770467 22010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:37.771035 22002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:37.771282 22002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:37.772457 22002 hybrid_clock.cc:648] HybridClock initialized: now 1790201497772450 us; error 38 us; skew 500 ppm
I20260923 22:11:37.773665 22002 webserver.cc:460] Webserver started at http://127.1.244.195:36535/ using document root <none> and password file <none>
I20260923 22:11:37.773912 22002 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:37.773983 22002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:37.774118 22002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:37.775180 22002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/data/instance:
uuid: "df770c8b2a08497ba69f08a5a6e91778"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "d3bb1836b9f43e87ce799c93e22bb1a1"
server_key_iv: "5e895c60069e3339643ddb798df04818"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.775605 22002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "df770c8b2a08497ba69f08a5a6e91778"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "d3bb1836b9f43e87ce799c93e22bb1a1"
server_key_iv: "5e895c60069e3339643ddb798df04818"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.776989 22002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:37.777765 22016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.777930 22002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:37.778028 22002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/wal
uuid: "df770c8b2a08497ba69f08a5a6e91778"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-b5jz"
server_key: "d3bb1836b9f43e87ce799c93e22bb1a1"
server_key_iv: "5e895c60069e3339643ddb798df04818"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.778112 22002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:37.817054 22002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:37.817447 22002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:37.817610 22002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:37.817868 22002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:37.818284 22002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:37.818343 22002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.818382 22002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:37.818414 22002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.825443 22002 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:33471
I20260923 22:11:37.825481 22129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:33471 every 8 connection(s)
I20260923 22:11:37.825809 22002 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:37.826452 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 22002
I20260923 22:11:37.826581 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:37.826792 2003 external_mini_cluster.cc:1428] Setting key f991321c93de14ade453b6b9c8019b8b
I20260923 22:11:37.831681 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.826547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:58663 (local address 127.1.244.254:38441)
0923 22:11:37.826607 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.826609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.826900 (+ 291us) server_negotiation.cc:407] Connection header received
0923 22:11:37.827678 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.827679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.827692 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.827761 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:37.828505 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.829018 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.830166 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.830362 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.830426 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.831373 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:37.831374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:37.831374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:37.831378 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:37.831393 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:37.831396 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:37.831440 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.831539 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.831615 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:11:37.831925 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.826534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38441 (local address 127.1.244.195:58663)
0923 22:11:37.826732 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.826780 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.827487 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.827853 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.827857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.827868 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:37.828421 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.828426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.829096 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.829097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.830085 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.830087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.830499 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.830500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.830616 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.831234 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:37.831234 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:37.831265 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:37.831291 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:37.831470 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:37.831476 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:37.831479 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.831599 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.831697 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260923 22:11:37.832492 22130 heartbeater.cc:347] Connected to a master server at 127.1.244.254:38441
I20260923 22:11:37.832576 22130 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:37.832713 22130 heartbeater.cc:511] Master 127.1.244.254:38441 requested a full tablet report, sending...
I20260923 22:11:37.833036 21680 ts_manager.cc:194] Registered new tserver with Master: df770c8b2a08497ba69f08a5a6e91778 (127.1.244.195:33471)
I20260923 22:11:37.833420 21680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.195:58663
I20260923 22:11:37.838884 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:37.842973 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.840309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56116 (local address 127.1.244.254:38441)
0923 22:11:37.840376 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.840378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.840496 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:11:37.840569 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.840570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.840581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.840633 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:37.840750 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.841205 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.841898 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.842071 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.842120 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.842684 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:37.842685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:37.842686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:37.842689 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:37.842703 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:37.842706 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:37.842749 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.842817 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.842839 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:11:37.844435 21680 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:37.846015 21680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:37.852835 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.848480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:33471 (local address 127.0.0.1:53348)
0923 22:11:37.849318 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.849326 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.849387 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.849576 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.849576 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.849578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:37.849613 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.849615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.850452 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.850453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.851802 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.851805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.852533 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.852534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.852608 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.852620 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.852696 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.852760 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":73,"thread_start_us":65,"threads_started":1}
I20260923 22:11:37.852851 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.848626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.1.244.195:33471)
0923 22:11:37.848823 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.848828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.849390 (+ 562us) server_negotiation.cc:407] Connection header received
0923 22:11:37.849456 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.849459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.849474 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.849577 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:37.849668 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.850358 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.851902 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.852416 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.852532 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.852631 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.852711 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.852760 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
I20260923 22:11:37.852952 22142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.848264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:35787 (local address 127.0.0.1:53018)
0923 22:11:37.848799 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.848808 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.848847 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.849182 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.849183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.849185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:37.849242 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.849244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.850501 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.850502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.851867 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.851869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.852696 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.852697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.852778 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.852782 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.852826 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.852869 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":83,"spinlock_wait_cycles":768,"thread_start_us":66,"threads_started":1}
I20260923 22:11:37.853284 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.848450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53018 (local address 127.1.244.194:35787)
0923 22:11:37.848962 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.848966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.848976 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:11:37.849015 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.849018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.849032 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.849108 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:37.849328 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.850400 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.851932 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.852593 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.853031 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.853124 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.853150 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.853194 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":93,"threads_started":1}
I20260923 22:11:37.853269 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.848381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59964 (local address 127.1.244.193:40153)
0923 22:11:37.848589 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.848594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.848775 (+ 181us) server_negotiation.cc:407] Connection header received
0923 22:11:37.848927 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.848930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.848948 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.849022 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:37.849192 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.850375 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.851849 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.852588 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.852666 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.852772 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.852991 (+ 219us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.853034 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":103,"threads_started":1}
I20260923 22:11:37.853900 22064 tablet_service.cc:1467] Processing CreateTablet for tablet d7b0650faac84c4fbdf834ab6968f118 (DEFAULT_TABLE table=test-table [id=bcc44814bdee4b3db66c55c70163c1f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:37.854243 22064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b0650faac84c4fbdf834ab6968f118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:37.853433 22139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.848264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:40153 (local address 127.0.0.1:59964)
0923 22:11:37.848646 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:37.848692 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:37.848774 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:37.849038 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:37.849041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:37.849044 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:37.849109 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.849112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.850477 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.850478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.851752 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:37.851755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.852804 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.852804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:37.852858 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.852861 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:37.852907 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:11:37.852937 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":33,"thread_start_us":71,"threads_started":1}
I20260923 22:11:37.857373 22144 tablet_bootstrap.cc:492] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: Bootstrap starting.
I20260923 22:11:37.857856 21802 tablet_service.cc:1467] Processing CreateTablet for tablet d7b0650faac84c4fbdf834ab6968f118 (DEFAULT_TABLE table=test-table [id=bcc44814bdee4b3db66c55c70163c1f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:37.858131 22144 tablet_bootstrap.cc:654] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:37.858217 21802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b0650faac84c4fbdf834ab6968f118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:37.858278 21933 tablet_service.cc:1467] Processing CreateTablet for tablet d7b0650faac84c4fbdf834ab6968f118 (DEFAULT_TABLE table=test-table [id=bcc44814bdee4b3db66c55c70163c1f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:37.858436 22144 log.cc:824] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:37.858774 21933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b0650faac84c4fbdf834ab6968f118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:37.861653 22146 tablet_bootstrap.cc:492] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732: Bootstrap starting.
I20260923 22:11:37.862196 22144 tablet_bootstrap.cc:492] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: No bootstrap required, opened a new log
I20260923 22:11:37.862253 22144 ts_tablet_manager.cc:1397] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 22:11:37.862334 22147 tablet_bootstrap.cc:492] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: Bootstrap starting.
I20260923 22:11:37.862890 22147 tablet_bootstrap.cc:654] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:37.863188 22147 log.cc:824] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:37.863876 22147 tablet_bootstrap.cc:492] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: No bootstrap required, opened a new log
I20260923 22:11:37.863963 22147 ts_tablet_manager.cc:1397] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:37.864559 22144 raft_consensus.cc:348] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } }
I20260923 22:11:37.864739 22144 raft_consensus.cc:374] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:37.864765 22144 raft_consensus.cc:729] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df770c8b2a08497ba69f08a5a6e91778, State: Initialized, Role: FOLLOWER
I20260923 22:11:37.864921 22144 consensus_queue.cc:260] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } }
I20260923 22:11:37.865222 22144 ts_tablet_manager.cc:1428] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:11:37.865350 22130 heartbeater.cc:503] Master 127.1.244.254:38441 was elected leader, sending a full tablet report...
I20260923 22:11:37.865888 22146 tablet_bootstrap.cc:654] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:37.866225 22146 log.cc:824] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:37.866648 22147 raft_consensus.cc:348] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } }
I20260923 22:11:37.866817 22147 raft_consensus.cc:374] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:37.866863 22147 raft_consensus.cc:729] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776b2fcf415a4ad5a09b3d8413856a85, State: Initialized, Role: FOLLOWER
I20260923 22:11:37.866987 22146 tablet_bootstrap.cc:492] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732: No bootstrap required, opened a new log
I20260923 22:11:37.866971 22147 consensus_queue.cc:260] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } }
I20260923 22:11:37.867056 22146 ts_tablet_manager.cc:1397] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 22:11:37.867215 22147 ts_tablet_manager.cc:1428] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:37.867352 21999 heartbeater.cc:503] Master 127.1.244.254:38441 was elected leader, sending a full tablet report...
I20260923 22:11:37.869354 22146 raft_consensus.cc:348] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } }
I20260923 22:11:37.869477 22146 raft_consensus.cc:374] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:37.869498 22146 raft_consensus.cc:729] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c018817db5e4842a5029d7fcdc3d732, State: Initialized, Role: FOLLOWER
I20260923 22:11:37.869668 22146 consensus_queue.cc:260] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } }
I20260923 22:11:37.869930 22146 ts_tablet_manager.cc:1428] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:37.869972 21868 heartbeater.cc:503] Master 127.1.244.254:38441 was elected leader, sending a full tablet report...
W20260923 22:11:37.930306 22000 tablet.cc:2369] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:11:38.057530 21869 tablet.cc:2369] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:11:38.076349 22131 tablet.cc:2369] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:38.079357 22152 raft_consensus.cc:482] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:38.079468 22152 raft_consensus.cc:504] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } }
I20260923 22:11:38.079833 22152 leader_election.cc:288] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776b2fcf415a4ad5a09b3d8413856a85 (127.1.244.194:35787), df770c8b2a08497ba69f08a5a6e91778 (127.1.244.195:33471)
I20260923 22:11:38.083091 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.080370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:36563 (local address 127.1.244.195:33471)
0923 22:11:38.080434 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.080436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.080445 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:11:38.080522 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.080523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.080535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.080580 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:38.080766 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.081289 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.082348 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.082711 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.082768 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.082775 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.082978 (+ 203us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.083020 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:11:38.083105 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.080131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:37231 (local address 127.1.244.194:35787)
0923 22:11:38.080196 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.080198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.080411 (+ 213us) server_negotiation.cc:407] Connection header received
0923 22:11:38.080447 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.080448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.080461 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.080527 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:38.080816 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.081301 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.082381 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.082733 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.082829 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.082837 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.083005 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.083043 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:11:38.083105 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.080273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:33471 (local address 127.1.244.193:36563)
0923 22:11:38.080366 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.080378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.080444 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.080612 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.080612 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.080614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:38.080679 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.080681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.081392 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.081393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.082256 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.082258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.082802 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.082803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.082895 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.082899 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.082942 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.083010 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":21,"thread_start_us":41,"threads_started":1}
I20260923 22:11:38.083110 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.080054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:35787 (local address 127.1.244.193:37231)
0923 22:11:38.080192 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.080205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.080271 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.080624 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.080625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.080627 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:38.080700 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.080702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.081429 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.081430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.082264 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.082266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.082850 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.082850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.082902 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.082906 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.082982 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.083020 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260923 22:11:38.083565 22084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b0650faac84c4fbdf834ab6968f118" candidate_uuid: "5c018817db5e4842a5029d7fcdc3d732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df770c8b2a08497ba69f08a5a6e91778" is_pre_election: true
I20260923 22:11:38.083730 22084 raft_consensus.cc:2454] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c018817db5e4842a5029d7fcdc3d732 in term 0.
I20260923 22:11:38.083757 21953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b0650faac84c4fbdf834ab6968f118" candidate_uuid: "5c018817db5e4842a5029d7fcdc3d732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776b2fcf415a4ad5a09b3d8413856a85" is_pre_election: true
I20260923 22:11:38.083906 21953 raft_consensus.cc:2454] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c018817db5e4842a5029d7fcdc3d732 in term 0.
I20260923 22:11:38.083968 21756 leader_election.cc:302] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c018817db5e4842a5029d7fcdc3d732, df770c8b2a08497ba69f08a5a6e91778; no voters:
I20260923 22:11:38.084127 22152 raft_consensus.cc:2781] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:38.084213 22152 raft_consensus.cc:482] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:38.084250 22152 raft_consensus.cc:3037] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:38.084975 22152 raft_consensus.cc:504] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } }
I20260923 22:11:38.085119 22152 leader_election.cc:288] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [CANDIDATE]: Term 1 election: Requested vote from peers 776b2fcf415a4ad5a09b3d8413856a85 (127.1.244.194:35787), df770c8b2a08497ba69f08a5a6e91778 (127.1.244.195:33471)
I20260923 22:11:38.085289 21953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b0650faac84c4fbdf834ab6968f118" candidate_uuid: "5c018817db5e4842a5029d7fcdc3d732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776b2fcf415a4ad5a09b3d8413856a85"
I20260923 22:11:38.085374 21953 raft_consensus.cc:3037] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:38.085407 22084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b0650faac84c4fbdf834ab6968f118" candidate_uuid: "5c018817db5e4842a5029d7fcdc3d732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df770c8b2a08497ba69f08a5a6e91778"
I20260923 22:11:38.085508 22084 raft_consensus.cc:3037] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:38.086155 21953 raft_consensus.cc:2454] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c018817db5e4842a5029d7fcdc3d732 in term 1.
I20260923 22:11:38.086243 22084 raft_consensus.cc:2454] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c018817db5e4842a5029d7fcdc3d732 in term 1.
I20260923 22:11:38.086412 21756 leader_election.cc:302] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c018817db5e4842a5029d7fcdc3d732, df770c8b2a08497ba69f08a5a6e91778; no voters:
I20260923 22:11:38.086544 22152 raft_consensus.cc:2781] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:38.086719 22152 raft_consensus.cc:686] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [term 1 LEADER]: Becoming Leader. State: Replica: 5c018817db5e4842a5029d7fcdc3d732, State: Running, Role: LEADER
I20260923 22:11:38.086833 22152 consensus_queue.cc:237] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } }
I20260923 22:11:38.087733 21680 catalog_manager.cc:5495] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c018817db5e4842a5029d7fcdc3d732 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "5c018817db5e4842a5029d7fcdc3d732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c018817db5e4842a5029d7fcdc3d732" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 } health_report { overall_health: UNKNOWN } } }
I20260923 22:11:38.136757 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.134445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59978 (local address 127.1.244.193:40153)
0923 22:11:38.134521 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.134523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.134531 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:11:38.134569 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.134570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.134590 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.134652 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:38.134768 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.135253 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.136051 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.136249 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.136309 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.136438 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:38.136541 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:38.136601 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.136632 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.136650 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 22:11:38.138837 21953 raft_consensus.cc:1261] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [term 1 FOLLOWER]: Refusing update from remote peer 5c018817db5e4842a5029d7fcdc3d732: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:38.138880 22084 raft_consensus.cc:1261] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [term 1 FOLLOWER]: Refusing update from remote peer 5c018817db5e4842a5029d7fcdc3d732: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:38.139293 22152 consensus_queue.cc:1035] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776b2fcf415a4ad5a09b3d8413856a85" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:38.139446 22155 consensus_queue.cc:1035] T d7b0650faac84c4fbdf834ab6968f118 P 5c018817db5e4842a5029d7fcdc3d732 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df770c8b2a08497ba69f08a5a6e91778" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:38.142166 22159 mvcc.cc:205] Tried to move back new op lower bound from 7332665336373501952 to 7332665336164409344. Current Snapshot: MvccSnapshot[applied={T|T < 7332665336373501952}]
I20260923 22:11:38.142647 22160 mvcc.cc:205] Tried to move back new op lower bound from 7332665336373501952 to 7332665336164409344. Current Snapshot: MvccSnapshot[applied={T|T < 7332665336373501952}]
I20260923 22:11:38.144131 21679 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:38.144213 21679 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:38.144955 21679 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 202c089baf8349ad8ddbfdc269bcc4f0: Sending DeleteTablet for 3 replicas of tablet d7b0650faac84c4fbdf834ab6968f118
I20260923 22:11:38.145304 22064 tablet_service.cc:1514] Processing DeleteTablet for tablet d7b0650faac84c4fbdf834ab6968f118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:38 UTC) from {username='slave'} at 127.0.0.1:53348
I20260923 22:11:38.145308 21933 tablet_service.cc:1514] Processing DeleteTablet for tablet d7b0650faac84c4fbdf834ab6968f118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:38 UTC) from {username='slave'} at 127.0.0.1:53018
I20260923 22:11:38.145397 21802 tablet_service.cc:1514] Processing DeleteTablet for tablet d7b0650faac84c4fbdf834ab6968f118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:38 UTC) from {username='slave'} at 127.0.0.1:59964
I20260923 22:11:38.145543 22168 tablet_replica.cc:330] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: stopping tablet replica
I20260923 22:11:38.145638 22168 raft_consensus.cc:2229] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:38.145740 22168 raft_consensus.cc:2258] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:38.145910 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 21740
I20260923 22:11:38.147135 22167 tablet_replica.cc:330] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: stopping tablet replica
I20260923 22:11:38.147241 22167 raft_consensus.cc:2229] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:38.147293 22168 ts_tablet_manager.cc:1905] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:38.147348 22167 raft_consensus.cc:2258] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:38.147759 22167 ts_tablet_manager.cc:1905] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:38.149051 22168 ts_tablet_manager.cc:1918] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:38.149142 22168 log.cc:1196] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-1/wal/wals/d7b0650faac84c4fbdf834ab6968f118
I20260923 22:11:38.149323 22167 ts_tablet_manager.cc:1918] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:38.149392 22167 log.cc:1196] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: Deleting WAL directory at /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201256447402-2003-0/minicluster-data/ts-2/wal/wals/d7b0650faac84c4fbdf834ab6968f118
I20260923 22:11:38.149456 22168 ts_tablet_manager.cc:1939] T d7b0650faac84c4fbdf834ab6968f118 P 776b2fcf415a4ad5a09b3d8413856a85: Deleting consensus metadata
I20260923 22:11:38.149624 22167 ts_tablet_manager.cc:1939] T d7b0650faac84c4fbdf834ab6968f118 P df770c8b2a08497ba69f08a5a6e91778: Deleting consensus metadata
I20260923 22:11:38.149923 21665 catalog_manager.cc:4841] TS 776b2fcf415a4ad5a09b3d8413856a85 (127.1.244.194:35787): tablet d7b0650faac84c4fbdf834ab6968f118 (table test-table [id=bcc44814bdee4b3db66c55c70163c1f9]) successfully deleted
I20260923 22:11:38.150033 21665 catalog_manager.cc:4841] TS df770c8b2a08497ba69f08a5a6e91778 (127.1.244.195:33471): tablet d7b0650faac84c4fbdf834ab6968f118 (table test-table [id=bcc44814bdee4b3db66c55c70163c1f9]) successfully deleted
W20260923 22:11:38.152045 21666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:40153 (error 108)
I20260923 22:11:38.152140 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 21871
I20260923 22:11:38.152974 22142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.152854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:40153 (local address 127.0.0.1:59984)
0923 22:11:38.152899 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.152941 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:40153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260923 22:11:38.153102 21666 catalog_manager.cc:4584] TS 5c018817db5e4842a5029d7fcdc3d732 (127.1.244.193:40153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d7b0650faac84c4fbdf834ab6968f118: Network error: Client connection negotiation failed: client connection to 127.1.244.193:40153: connect: Connection refused (error 111)
I20260923 22:11:38.157749 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 22002
I20260923 22:11:38.162127 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 21649
2026-09-23T22:11:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (950 ms)
[----------] 6 tests from EncryptionPolicyTest (8788 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:38 dist-test-slave-b5jz krb5kdc[22173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:38 dist-test-slave-b5jz krb5kdc[22173](info): set up 2 sockets
Sep 23 22:11:38 dist-test-slave-b5jz krb5kdc[22173](info): commencing operation
krb5kdc: starting...
W20260923 22:11:39.195710 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:39 dist-test-slave-b5jz krb5kdc[22173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:39.311829 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:39.396263 22186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:39.396467 22186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:39.396490 22186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:39.398015 22186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:39.398070 22186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:39.398085 22186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:39.398098 22186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:39.398111 22186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:39.399677 22186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:39.399888 22186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:39.400058 22186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:39.404724 22191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:39.404758 22192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:39.404735 22194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:39.404974 22186 server_base.cc:1034] running on GCE node
I20260923 22:11:39.405352 22186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:39.405651 22186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:39.406826 22186 hybrid_clock.cc:648] HybridClock initialized: now 1790201499406793 us; error 53 us; skew 500 ppm
Sep 23 22:11:39 dist-test-slave-b5jz krb5kdc[22173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201499, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:39.409324 22186 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:39.409776 22186 webserver.cc:460] Webserver started at http://127.0.0.1:42515/ using document root <none> and password file <none>
I20260923 22:11:39.409976 22186 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:39.410017 22186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:39.410111 22186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:39.411095 22186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "43fc07f736694cc49facfcd96f71c826"
format_stamp: "Formatted at 2026-09-23 22:11:39 on dist-test-slave-b5jz"
server_key: "e11cad6d73eb6700976ed003d4dfd99a"
server_key_iv: "d213735bd420a06ca8ae00efa6f3d364"
server_key_version: "encryptionkey@0"
I20260923 22:11:39.411381 22186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "43fc07f736694cc49facfcd96f71c826"
format_stamp: "Formatted at 2026-09-23 22:11:39 on dist-test-slave-b5jz"
server_key: "e11cad6d73eb6700976ed003d4dfd99a"
server_key_iv: "d213735bd420a06ca8ae00efa6f3d364"
server_key_version: "encryptionkey@0"
I20260923 22:11:39.412714 22186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:11:39.413460 22201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:39.413662 22186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:39.413753 22186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "43fc07f736694cc49facfcd96f71c826"
format_stamp: "Formatted at 2026-09-23 22:11:39 on dist-test-slave-b5jz"
server_key: "e11cad6d73eb6700976ed003d4dfd99a"
server_key_iv: "d213735bd420a06ca8ae00efa6f3d364"
server_key_version: "encryptionkey@0"
I20260923 22:11:39.413806 22186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:39.430732 22186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:39.433085 22186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:39.433197 22186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:39.437402 22186 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46419
I20260923 22:11:39.437479 22253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46419 every 8 connection(s)
I20260923 22:11:39.437784 22186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:39.438756 22254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:39.441040 22254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826: Bootstrap starting.
I20260923 22:11:39.441610 22254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:39.441907 22254 log.cc:824] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:39.442582 22254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826: No bootstrap required, opened a new log
I20260923 22:11:39.444172 22254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fc07f736694cc49facfcd96f71c826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } }
I20260923 22:11:39.444309 22254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:39.444331 22254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43fc07f736694cc49facfcd96f71c826, State: Initialized, Role: FOLLOWER
I20260923 22:11:39.444506 22254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fc07f736694cc49facfcd96f71c826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } }
I20260923 22:11:39.444597 22254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:39.444641 22254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:39.444692 22254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:39.445210 22254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fc07f736694cc49facfcd96f71c826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } }
I20260923 22:11:39.445323 22254 leader_election.cc:302] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43fc07f736694cc49facfcd96f71c826; no voters:
I20260923 22:11:39.445505 22254 leader_election.cc:288] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:39.445544 22257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:39.445729 22257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [term 1 LEADER]: Becoming Leader. State: Replica: 43fc07f736694cc49facfcd96f71c826, State: Running, Role: LEADER
I20260923 22:11:39.445762 22254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:39.445815 22257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fc07f736694cc49facfcd96f71c826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } }
I20260923 22:11:39.446192 22259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43fc07f736694cc49facfcd96f71c826. Latest consensus state: current_term: 1 leader_uuid: "43fc07f736694cc49facfcd96f71c826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fc07f736694cc49facfcd96f71c826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } } }
I20260923 22:11:39.446324 22258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43fc07f736694cc49facfcd96f71c826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fc07f736694cc49facfcd96f71c826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46419 } } }
I20260923 22:11:39.446412 22258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:39.446421 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 22186
I20260923 22:11:39.446353 22259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:39.446535 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:39.446627 22261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:39.446734 2003 external_mini_cluster.cc:1428] Setting key cb36874759c14d2abd44fa29fef5f3b0
I20260923 22:11:39.447140 22261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:39.449527 22261 catalog_manager.cc:1347] Generated new cluster ID: 7dde9e6aa1754c0c8490c0089616b601
I20260923 22:11:39.449590 22261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:39 dist-test-slave-b5jz krb5kdc[22173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:11:39.460261 22261 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:39.460966 22261 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:39.461278 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.447404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38463 (local address 127.0.0.1:46419)
0923 22:11:39.448087 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.448098 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.448129 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:11:39.448230 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.448236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.448286 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.448427 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:39.449215 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.450279 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.451049 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.451474 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.451624 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.457727 (+ 6103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:39.457735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:39.457735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:39.457747 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:39.459403 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:39.459857 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:39.459860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:39.459860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:39.459965 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:39.460219 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:39.460220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:39.460220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:39.460512 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:39.460696 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.460792 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.460929 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":636,"thread_start_us":67,"threads_started":1}
I20260923 22:11:39.462287 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:39.468403 22261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43fc07f736694cc49facfcd96f71c826: Generated new TSK 0
I20260923 22:11:39.468578 22261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:39.474069 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.463822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58655 (local address 127.0.0.1:46419)
0923 22:11:39.463859 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.463862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.467717 (+ 3855us) server_negotiation.cc:407] Connection header received
0923 22:11:39.467781 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.467782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.467792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.467861 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:39.468450 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.469375 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.470123 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.470329 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.470405 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.471613 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:39.471614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:39.471615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:39.471622 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:39.472914 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:39.473285 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:39.473286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:39.473287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:39.473343 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:39.473568 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:39.473569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:39.473569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:39.473681 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:39.473796 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.473909 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.473977 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:11:39.475893 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 22186
W20260923 22:11:39.480201 22171 connection.cc:535] client connection to 127.0.0.1:46419 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46419): Connection reset by peer (error 104)
2026-09-23T22:11:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:39 dist-test-slave-b5jz krb5kdc[22285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:39 dist-test-slave-b5jz krb5kdc[22285](info): set up 2 sockets
Sep 23 22:11:39 dist-test-slave-b5jz krb5kdc[22285](info): commencing operation
krb5kdc: starting...
W20260923 22:11:40.519721 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:40 dist-test-slave-b5jz krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:40.638480 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44901
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:40.720924 22298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:40.721158 22298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:40.721199 22298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:40.722327 22298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:40.722402 22298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:40.722426 22298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:40.722446 22298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:40.722474 22298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:40.723940 22298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44901
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:40.724215 22298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:40.724440 22298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:40.729249 22304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:40.729255 22303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:40.729264 22306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:40.729286 22298 server_base.cc:1034] running on GCE node
I20260923 22:11:40.729696 22298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:40.729987 22298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:40.731133 22298 hybrid_clock.cc:648] HybridClock initialized: now 1790201500731122 us; error 29 us; skew 500 ppm
Sep 23 22:11:40 dist-test-slave-b5jz krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201500, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:40.737399 22298 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:40.737884 22298 webserver.cc:460] Webserver started at http://127.0.0.1:40671/ using document root <none> and password file <none>
I20260923 22:11:40.738091 22298 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:40.738147 22298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:40.738238 22298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:40.739249 22298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "14844cf03e404641840f6bff252ee263"
format_stamp: "Formatted at 2026-09-23 22:11:40 on dist-test-slave-b5jz"
server_key: "025ba2ed6d5cc8005a64a29eb27312e5"
server_key_iv: "733ab242f9f5e6015a8c3ae0dceca407"
server_key_version: "encryptionkey@0"
I20260923 22:11:40.739596 22298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "14844cf03e404641840f6bff252ee263"
format_stamp: "Formatted at 2026-09-23 22:11:40 on dist-test-slave-b5jz"
server_key: "025ba2ed6d5cc8005a64a29eb27312e5"
server_key_iv: "733ab242f9f5e6015a8c3ae0dceca407"
server_key_version: "encryptionkey@0"
I20260923 22:11:40.740914 22298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:40.741626 22313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:40.741808 22298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:40.741904 22298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "14844cf03e404641840f6bff252ee263"
format_stamp: "Formatted at 2026-09-23 22:11:40 on dist-test-slave-b5jz"
server_key: "025ba2ed6d5cc8005a64a29eb27312e5"
server_key_iv: "733ab242f9f5e6015a8c3ae0dceca407"
server_key_version: "encryptionkey@0"
I20260923 22:11:40.741983 22298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:40.770890 22298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:40.773819 22298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:40.773999 22298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:40.778733 22298 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44901
I20260923 22:11:40.778760 22365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44901 every 8 connection(s)
I20260923 22:11:40.779147 22298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:40.780229 22366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:40.782829 22366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263: Bootstrap starting.
I20260923 22:11:40.783450 22366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:40.783771 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 22298
I20260923 22:11:40.783783 22366 log.cc:824] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:40.783901 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:40.784155 2003 external_mini_cluster.cc:1428] Setting key 287188c74776e22a704e88b4985938cf
I20260923 22:11:40.784502 22366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263: No bootstrap required, opened a new log
I20260923 22:11:40.786171 22366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14844cf03e404641840f6bff252ee263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44901 } }
I20260923 22:11:40.786541 22366 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:40.786664 22366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14844cf03e404641840f6bff252ee263, State: Initialized, Role: FOLLOWER
I20260923 22:11:40.786834 22366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14844cf03e404641840f6bff252ee263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44901 } }
I20260923 22:11:40.786931 22366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:40.786968 22366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:40.787065 22366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:40.787949 22366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14844cf03e404641840f6bff252ee263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44901 } }
I20260923 22:11:40.788059 22366 leader_election.cc:302] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14844cf03e404641840f6bff252ee263; no voters:
I20260923 22:11:40.788219 22366 leader_election.cc:288] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:40.788341 22371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:40.788446 22366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:40.788529 22371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [term 1 LEADER]: Becoming Leader. State: Replica: 14844cf03e404641840f6bff252ee263, State: Running, Role: LEADER
I20260923 22:11:40.788654 22371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14844cf03e404641840f6bff252ee263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44901 } }
I20260923 22:11:40.789124 22372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14844cf03e404641840f6bff252ee263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14844cf03e404641840f6bff252ee263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44901 } } }
I20260923 22:11:40.789212 22372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:40.789135 22373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14844cf03e404641840f6bff252ee263. Latest consensus state: current_term: 1 leader_uuid: "14844cf03e404641840f6bff252ee263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14844cf03e404641840f6bff252ee263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44901 } } }
I20260923 22:11:40.789422 22380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:40.789422 22373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:40.789974 22380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:40.791877 22380 catalog_manager.cc:1347] Generated new cluster ID: 302fd942db654e249fe074432c70a824
I20260923 22:11:40.791946 22380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:40.792459 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:40.784692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48563 (local address 127.0.0.1:44901)
0923 22:11:40.784863 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:40.784874 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:40.784909 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:11:40.786166 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:40.786174 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:40.786229 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:40.786406 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:40.786674 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:40.788578 (+ 1904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:40.789859 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:40.790710 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:40.790823 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:40.791598 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:40.791602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:40.791603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:40.791823 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:40.791862 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:40.791877 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:40.791967 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:40.792229 (+ 262us) server_negotiation.cc:299] Negotiation successful
0923 22:11:40.792255 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260923 22:11:40.793570 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:40.798683 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:40.795114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48065 (local address 127.0.0.1:44901)
0923 22:11:40.795158 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:40.795160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:40.795263 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:11:40.795309 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:40.795310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:40.795320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:40.795367 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:40.795520 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:40.796253 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:40.797356 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:40.797682 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:40.797741 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:40.798443 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:40.798444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:40.798444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:40.798449 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:40.798463 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:40.798467 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:40.798524 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:40.798603 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:11:40.798614 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:11:40.811637 22380 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:40.812181 22380 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:40.823168 22380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14844cf03e404641840f6bff252ee263: Generated new TSK 0
I20260923 22:11:40.823393 22380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:40.834667 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 22298
W20260923 22:11:40.838996 22283 connection.cc:535] client connection to 127.0.0.1:44901 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44901): Connection reset by peer (error 104)
2026-09-23T22:11:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1358 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:40 dist-test-slave-b5jz krb5kdc[22397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:40 dist-test-slave-b5jz krb5kdc[22397](info): set up 2 sockets
Sep 23 22:11:40 dist-test-slave-b5jz krb5kdc[22397](info): commencing operation
krb5kdc: starting...
W20260923 22:11:41.876518 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:41 dist-test-slave-b5jz krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:41.998445 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:42.079119 22410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:42.079315 22410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:42.079339 22410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:42.080452 22410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:42.080488 22410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:42.080502 22410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:42.080515 22410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:42.080528 22410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:42.082067 22410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:42.082260 22410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:42.082427 22410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:42.087185 22418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:42.087189 22416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:42.087191 22415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:42.087471 22410 server_base.cc:1034] running on GCE node
I20260923 22:11:42.087762 22410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:42.088075 22410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:42.089250 22410 hybrid_clock.cc:648] HybridClock initialized: now 1790201502089236 us; error 29 us; skew 500 ppm
Sep 23 22:11:42 dist-test-slave-b5jz krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201502, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:42.091717 22410 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:42.092232 22410 webserver.cc:460] Webserver started at http://127.0.0.1:32909/ using document root <none> and password file <none>
I20260923 22:11:42.092495 22410 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:42.092566 22410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:42.092700 22410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:42.093746 22410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "0d08a180d26f4a6e8980173e5a808965"
format_stamp: "Formatted at 2026-09-23 22:11:42 on dist-test-slave-b5jz"
server_key: "d4b6956709f80826f9ded22092cb9701"
server_key_iv: "fc3cc63a6b33936b3da46da66ec0c174"
server_key_version: "encryptionkey@0"
I20260923 22:11:42.094120 22410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "0d08a180d26f4a6e8980173e5a808965"
format_stamp: "Formatted at 2026-09-23 22:11:42 on dist-test-slave-b5jz"
server_key: "d4b6956709f80826f9ded22092cb9701"
server_key_iv: "fc3cc63a6b33936b3da46da66ec0c174"
server_key_version: "encryptionkey@0"
I20260923 22:11:42.095412 22410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:42.096184 22425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:42.096369 22410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:42.096475 22410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "0d08a180d26f4a6e8980173e5a808965"
format_stamp: "Formatted at 2026-09-23 22:11:42 on dist-test-slave-b5jz"
server_key: "d4b6956709f80826f9ded22092cb9701"
server_key_iv: "fc3cc63a6b33936b3da46da66ec0c174"
server_key_version: "encryptionkey@0"
I20260923 22:11:42.096562 22410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:42.099822 22410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:42.101631 22410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:42.101797 22410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:42.105836 22410 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43967
I20260923 22:11:42.105866 22477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43967 every 8 connection(s)
I20260923 22:11:42.106191 22410 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:42.107086 22478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:42.109302 22478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965: Bootstrap starting.
I20260923 22:11:42.109805 22478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:42.110026 22478 log.cc:824] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:42.110672 22478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965: No bootstrap required, opened a new log
I20260923 22:11:42.112285 22478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d08a180d26f4a6e8980173e5a808965" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260923 22:11:42.112418 22478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:42.112440 22478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d08a180d26f4a6e8980173e5a808965, State: Initialized, Role: FOLLOWER
I20260923 22:11:42.112576 22478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d08a180d26f4a6e8980173e5a808965" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260923 22:11:42.112654 22478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:42.112679 22478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:42.112707 22478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:42.113212 22478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d08a180d26f4a6e8980173e5a808965" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260923 22:11:42.113297 22478 leader_election.cc:302] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d08a180d26f4a6e8980173e5a808965; no voters:
I20260923 22:11:42.113410 22478 leader_election.cc:288] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:42.113488 22481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:42.113657 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 22410
I20260923 22:11:42.113684 22481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [term 1 LEADER]: Becoming Leader. State: Replica: 0d08a180d26f4a6e8980173e5a808965, State: Running, Role: LEADER
I20260923 22:11:42.113790 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:42.113689 22478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:42.113790 22481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d08a180d26f4a6e8980173e5a808965" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260923 22:11:42.114017 2003 external_mini_cluster.cc:1428] Setting key fe9cbf4d23d2220cd3f4f80ab8e1bd2b
I20260923 22:11:42.114209 22483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d08a180d26f4a6e8980173e5a808965. Latest consensus state: current_term: 1 leader_uuid: "0d08a180d26f4a6e8980173e5a808965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d08a180d26f4a6e8980173e5a808965" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } } }
I20260923 22:11:42.114281 22483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:42.114344 22482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d08a180d26f4a6e8980173e5a808965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d08a180d26f4a6e8980173e5a808965" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } } }
I20260923 22:11:42.114426 22482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:42.114909 22488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:42.115489 22488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:42.116571 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:42.114756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59235 (local address 127.0.0.1:43967)
0923 22:11:42.114943 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:42.114953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:42.115020 (+ 67us) server_negotiation.cc:407] Connection header received
0923 22:11:42.115140 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:42.115148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:42.115189 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:42.115670 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:42.116131 (+ 461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:42.116134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:42.116134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:42.116137 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:42.116164 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:42.116173 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:42.116259 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:42.116402 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 22:11:42.116437 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":109,"threads_started":1}
I20260923 22:11:42.117549 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:42.119064 22488 catalog_manager.cc:1347] Generated new cluster ID: a70f8945c1e7491587f05a06e34121c3
I20260923 22:11:42.119118 22488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:42.120893 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:42.119649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60547 (local address 127.0.0.1:43967)
0923 22:11:42.119700 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:42.119702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:42.119866 (+ 164us) server_negotiation.cc:407] Connection header received
0923 22:11:42.119937 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:42.119938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:42.119943 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:42.119998 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:42.120644 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:42.120645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:42.120646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:42.120650 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:42.120665 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:42.120669 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:42.120726 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:42.120782 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:11:42.120794 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:11:42.142911 22488 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:42.143134 22488 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:42.152081 22488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d08a180d26f4a6e8980173e5a808965: Generated new TSK 0
I20260923 22:11:42.152408 22488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:42.154788 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 22410
2026-09-23T22:11:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:42 dist-test-slave-b5jz krb5kdc[22509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:42 dist-test-slave-b5jz krb5kdc[22509](info): set up 2 sockets
Sep 23 22:11:42 dist-test-slave-b5jz krb5kdc[22509](info): commencing operation
krb5kdc: starting...
W20260923 22:11:43.179766 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:43 dist-test-slave-b5jz krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:43.298177 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40953
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:43.379103 22522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:43.379330 22522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:43.379369 22522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:43.380591 22522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:43.380661 22522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:43.380685 22522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:43.380708 22522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:43.380735 22522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:43.382189 22522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:43.382470 22522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:43.382696 22522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:43.387949 22528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:43.387945 22527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:43.387941 22530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:43.388015 22522 server_base.cc:1034] running on GCE node
I20260923 22:11:43.388387 22522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:43.388706 22522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:43.389878 22522 hybrid_clock.cc:648] HybridClock initialized: now 1790201503389859 us; error 32 us; skew 500 ppm
Sep 23 22:11:43 dist-test-slave-b5jz krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201503, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:43.392405 22522 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:43.392841 22522 webserver.cc:460] Webserver started at http://127.0.0.1:45211/ using document root <none> and password file <none>
I20260923 22:11:43.393043 22522 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:43.393085 22522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:43.393174 22522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:43.394122 22522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "d3328baa89b148dcb85a3f2ed6e34f79"
format_stamp: "Formatted at 2026-09-23 22:11:43 on dist-test-slave-b5jz"
server_key: "7a5741d3494360bf74acdc3b0cf7efec"
server_key_iv: "7c65cddc149bb845ea988e80faef4a90"
server_key_version: "encryptionkey@0"
I20260923 22:11:43.394412 22522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "d3328baa89b148dcb85a3f2ed6e34f79"
format_stamp: "Formatted at 2026-09-23 22:11:43 on dist-test-slave-b5jz"
server_key: "7a5741d3494360bf74acdc3b0cf7efec"
server_key_iv: "7c65cddc149bb845ea988e80faef4a90"
server_key_version: "encryptionkey@0"
I20260923 22:11:43.395689 22522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:11:43.396396 22537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:43.396581 22522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:11:43.396672 22522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "d3328baa89b148dcb85a3f2ed6e34f79"
format_stamp: "Formatted at 2026-09-23 22:11:43 on dist-test-slave-b5jz"
server_key: "7a5741d3494360bf74acdc3b0cf7efec"
server_key_iv: "7c65cddc149bb845ea988e80faef4a90"
server_key_version: "encryptionkey@0"
I20260923 22:11:43.396739 22522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:43.410822 22522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:43.412905 22522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:43.413084 22522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:43.416859 22522 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40953
I20260923 22:11:43.416945 22589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40953 every 8 connection(s)
I20260923 22:11:43.417265 22522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:43.418263 22590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:43.420491 22590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79: Bootstrap starting.
I20260923 22:11:43.421058 22590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:43.421342 22590 log.cc:824] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:43.422045 22590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79: No bootstrap required, opened a new log
I20260923 22:11:43.422811 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 22522
I20260923 22:11:43.422956 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:43.423166 2003 external_mini_cluster.cc:1428] Setting key 507d6bf963694a955e86f61126ddc5c6
I20260923 22:11:43.424239 22590 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3328baa89b148dcb85a3f2ed6e34f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40953 } }
I20260923 22:11:43.424410 22590 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:43.424448 22590 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3328baa89b148dcb85a3f2ed6e34f79, State: Initialized, Role: FOLLOWER
I20260923 22:11:43.424615 22590 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3328baa89b148dcb85a3f2ed6e34f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40953 } }
I20260923 22:11:43.424705 22590 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:43.424772 22590 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:43.424824 22590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:43.425844 22590 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3328baa89b148dcb85a3f2ed6e34f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40953 } }
I20260923 22:11:43.425951 22590 leader_election.cc:302] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3328baa89b148dcb85a3f2ed6e34f79; no voters:
I20260923 22:11:43.426126 22590 leader_election.cc:288] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:43.426216 22595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:43.426347 22590 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:43.426389 22595 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [term 1 LEADER]: Becoming Leader. State: Replica: d3328baa89b148dcb85a3f2ed6e34f79, State: Running, Role: LEADER
I20260923 22:11:43.426553 22595 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3328baa89b148dcb85a3f2ed6e34f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40953 } }
I20260923 22:11:43.426954 22595 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3328baa89b148dcb85a3f2ed6e34f79. Latest consensus state: current_term: 1 leader_uuid: "d3328baa89b148dcb85a3f2ed6e34f79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3328baa89b148dcb85a3f2ed6e34f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40953 } } }
I20260923 22:11:43.427083 22595 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:43.427671 22604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:43.428568 22604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:43.428712 22597 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3328baa89b148dcb85a3f2ed6e34f79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3328baa89b148dcb85a3f2ed6e34f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40953 } } }
I20260923 22:11:43.428779 22597 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:43.430214 22604 catalog_manager.cc:1347] Generated new cluster ID: eaac6ba532f54d0fb28214a5bf34add1
I20260923 22:11:43.430271 22604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:43 dist-test-slave-b5jz krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:11:43.434931 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:43.423711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39767 (local address 127.0.0.1:40953)
0923 22:11:43.423904 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:43.423915 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:43.423952 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:11:43.424199 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:43.424208 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:43.424270 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:43.424419 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:43.425207 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:43.426621 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:43.427742 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:43.428264 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:43.428382 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:43.431216 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:43.431219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:43.431220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:43.431232 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:43.433316 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:43.433698 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:43.433701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:43.433702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:43.433834 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:43.434048 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:43.434049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:43.434049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:43.434346 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:43.434522 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:43.434588 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:11:43.434719 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260923 22:11:43.436156 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:43.443604 22604 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:43.444325 22604 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:43.444697 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:43.437506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43121 (local address 127.0.0.1:40953)
0923 22:11:43.437588 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:43.437590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:43.437752 (+ 162us) server_negotiation.cc:407] Connection header received
0923 22:11:43.437778 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:43.437779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:43.437790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:43.437862 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:43.438430 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:43.439187 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:43.440181 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:43.440508 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:43.440591 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:43.441877 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:43.441878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:43.441878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:43.441885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:43.443471 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:43.443898 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:43.443899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:43.443900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:43.443963 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:43.444166 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:43.444167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:43.444167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:43.444281 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:43.444400 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:43.444490 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:11:43.444582 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 22:11:43.450583 22604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3328baa89b148dcb85a3f2ed6e34f79: Generated new TSK 0
I20260923 22:11:43.450767 22604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:43.452605 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 22522
2026-09-23T22:11:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:43 dist-test-slave-b5jz krb5kdc[22621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:43 dist-test-slave-b5jz krb5kdc[22621](info): set up 2 sockets
Sep 23 22:11:43 dist-test-slave-b5jz krb5kdc[22621](info): commencing operation
krb5kdc: starting...
W20260923 22:11:44.491667 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:44 dist-test-slave-b5jz krb5kdc[22621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:44.610378 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:44.690493 22634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:44.690724 22634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:44.690749 22634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:44.692317 22634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:44.692373 22634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:44.692389 22634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:44.692401 22634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:44.692476 22634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:44.694139 22634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:44.694415 22634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:44.694664 22634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:44.700104 22642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:44.700138 22639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:44.700120 22640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:44.700371 22634 server_base.cc:1034] running on GCE node
I20260923 22:11:44.700610 22634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:44.700958 22634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:44.702134 22634 hybrid_clock.cc:648] HybridClock initialized: now 1790201504702121 us; error 28 us; skew 500 ppm
Sep 23 22:11:44 dist-test-slave-b5jz krb5kdc[22621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201504, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:44.704672 22634 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:44.705199 22634 webserver.cc:460] Webserver started at http://127.0.0.1:38667/ using document root <none> and password file <none>
I20260923 22:11:44.705446 22634 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:44.705520 22634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:44.705677 22634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:44.707165 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "bfbb5d814ea44a3a9447ef324d5ee453"
format_stamp: "Formatted at 2026-09-23 22:11:44 on dist-test-slave-b5jz"
server_key: "65982993dbc174ef6f230d8379b37a81"
server_key_iv: "44a16bed6246c24086c71b85ea3fa567"
server_key_version: "encryptionkey@0"
I20260923 22:11:44.707695 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "bfbb5d814ea44a3a9447ef324d5ee453"
format_stamp: "Formatted at 2026-09-23 22:11:44 on dist-test-slave-b5jz"
server_key: "65982993dbc174ef6f230d8379b37a81"
server_key_iv: "44a16bed6246c24086c71b85ea3fa567"
server_key_version: "encryptionkey@0"
I20260923 22:11:44.709066 22634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:44.709856 22649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:44.710050 22634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:44.710182 22634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "bfbb5d814ea44a3a9447ef324d5ee453"
format_stamp: "Formatted at 2026-09-23 22:11:44 on dist-test-slave-b5jz"
server_key: "65982993dbc174ef6f230d8379b37a81"
server_key_iv: "44a16bed6246c24086c71b85ea3fa567"
server_key_version: "encryptionkey@0"
I20260923 22:11:44.710275 22634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:44.729756 22634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:44.731799 22634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:44.731976 22634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:44.736323 22634 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41087
I20260923 22:11:44.736356 22701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41087 every 8 connection(s)
I20260923 22:11:44.736742 22634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:44.737723 22702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:44.739974 22702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453: Bootstrap starting.
I20260923 22:11:44.740530 22702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:44.740774 22702 log.cc:824] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:44.741466 22702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453: No bootstrap required, opened a new log
I20260923 22:11:44.742954 22702 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbb5d814ea44a3a9447ef324d5ee453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260923 22:11:44.743093 22702 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:44.743115 22702 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbb5d814ea44a3a9447ef324d5ee453, State: Initialized, Role: FOLLOWER
I20260923 22:11:44.743216 22702 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbb5d814ea44a3a9447ef324d5ee453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260923 22:11:44.743314 22702 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:44.743353 22702 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:44.743407 22702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:44.744000 22702 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbb5d814ea44a3a9447ef324d5ee453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260923 22:11:44.744124 22702 leader_election.cc:302] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfbb5d814ea44a3a9447ef324d5ee453; no voters:
I20260923 22:11:44.744308 22702 leader_election.cc:288] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:44.744359 22705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:44.744568 22702 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:44.744639 22705 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [term 1 LEADER]: Becoming Leader. State: Replica: bfbb5d814ea44a3a9447ef324d5ee453, State: Running, Role: LEADER
I20260923 22:11:44.744755 22705 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbb5d814ea44a3a9447ef324d5ee453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260923 22:11:44.745179 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 22634
I20260923 22:11:44.745317 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:44.745429 22707 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbb5d814ea44a3a9447ef324d5ee453. Latest consensus state: current_term: 1 leader_uuid: "bfbb5d814ea44a3a9447ef324d5ee453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbb5d814ea44a3a9447ef324d5ee453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } } }
I20260923 22:11:44.745515 2003 external_mini_cluster.cc:1428] Setting key 4fb203b9f1eb5ec5450927a9539950ab
I20260923 22:11:44.745515 22707 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:44.745735 22706 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfbb5d814ea44a3a9447ef324d5ee453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbb5d814ea44a3a9447ef324d5ee453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } } }
I20260923 22:11:44.745803 22706 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:44.745911 22715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:44.746766 22715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:44.748488 22715 catalog_manager.cc:1347] Generated new cluster ID: 4930850960b649e5899ae9dba9388672
I20260923 22:11:44.748548 22715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:44.753331 22722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:44.746228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37661 (local address 127.0.0.1:41087)
0923 22:11:44.746408 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:44.746418 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:44.746455 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:11:44.747734 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:44.747743 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:44.747807 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:44.747952 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:44.748441 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:44.749830 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:44.750968 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:44.751592 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:44.751688 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:44.752452 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:44.752456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:44.752456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:44.752651 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:44.752687 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:44.752701 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:44.752771 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:44.752881 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:11:44.752942 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":52,"threads_started":1}
I20260923 22:11:44.754312 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:44.758036 22715 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:44.758646 22715 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:44.759174 22722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:44.755888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38277 (local address 127.0.0.1:41087)
0923 22:11:44.755927 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:44.755929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:44.755938 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:11:44.755961 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:44.755961 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:44.755971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:44.756034 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:44.756162 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:44.756909 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:44.757860 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:44.758165 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:44.758238 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:44.758918 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:44.758920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:44.758920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:44.758925 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:44.758940 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:44.758944 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:44.759004 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:44.759063 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 22:11:44.759101 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:11:44.765587 22715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfbb5d814ea44a3a9447ef324d5ee453: Generated new TSK 0
I20260923 22:11:44.765762 22715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:44.770470 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 22634
2026-09-23T22:11:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:44 dist-test-slave-b5jz krb5kdc[22733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:44 dist-test-slave-b5jz krb5kdc[22733](info): set up 2 sockets
Sep 23 22:11:44 dist-test-slave-b5jz krb5kdc[22733](info): commencing operation
krb5kdc: starting...
W20260923 22:11:45.803529 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:45 dist-test-slave-b5jz krb5kdc[22733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:45.922587 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38081
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:46.002394 22746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:46.002615 22746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:46.002640 22746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:46.003754 22746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:46.003793 22746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:46.003806 22746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:46.003819 22746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:46.003839 22746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:46.005261 22746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:46.005452 22746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:46.005621 22746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:46.010422 22752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:46.010443 22751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:46.010429 22754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:46.010447 22746 server_base.cc:1034] running on GCE node
I20260923 22:11:46.010785 22746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:46.011106 22746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:46.012315 22746 hybrid_clock.cc:648] HybridClock initialized: now 1790201506012307 us; error 42 us; skew 500 ppm
Sep 23 22:11:46 dist-test-slave-b5jz krb5kdc[22733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201506, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:46.018507 22746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:46.018996 22746 webserver.cc:460] Webserver started at http://127.0.0.1:41791/ using document root <none> and password file <none>
I20260923 22:11:46.019228 22746 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:46.019299 22746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:46.019457 22746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:46.020545 22746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835"
format_stamp: "Formatted at 2026-09-23 22:11:46 on dist-test-slave-b5jz"
server_key: "1c74e2c960c24204237bf186e96c4b74"
server_key_iv: "1f6585c300ae262bf0f5a0d08e28d0e8"
server_key_version: "encryptionkey@0"
I20260923 22:11:46.020932 22746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835"
format_stamp: "Formatted at 2026-09-23 22:11:46 on dist-test-slave-b5jz"
server_key: "1c74e2c960c24204237bf186e96c4b74"
server_key_iv: "1f6585c300ae262bf0f5a0d08e28d0e8"
server_key_version: "encryptionkey@0"
I20260923 22:11:46.022293 22746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:46.023025 22761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:46.023320 22746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:46.023447 22746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835"
format_stamp: "Formatted at 2026-09-23 22:11:46 on dist-test-slave-b5jz"
server_key: "1c74e2c960c24204237bf186e96c4b74"
server_key_iv: "1f6585c300ae262bf0f5a0d08e28d0e8"
server_key_version: "encryptionkey@0"
I20260923 22:11:46.023530 22746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:46.026856 22746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:46.028781 22746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:46.028959 22746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:46.033115 22746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38081
I20260923 22:11:46.033146 22813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38081 every 8 connection(s)
I20260923 22:11:46.033483 22746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:46.034402 22814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:46.036746 22814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835: Bootstrap starting.
I20260923 22:11:46.036875 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 22746
I20260923 22:11:46.037002 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:46.037189 2003 external_mini_cluster.cc:1428] Setting key 365ec8e34ae8682e0951dbacc346615e
I20260923 22:11:46.037460 22814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:46.037812 22814 log.cc:824] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:46.038537 22814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835: No bootstrap required, opened a new log
I20260923 22:11:46.039800 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:46.037680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48065 (local address 127.0.0.1:38081)
0923 22:11:46.037898 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:46.037907 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:46.037938 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:11:46.038523 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:46.038532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:46.038567 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:46.038729 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:46.039350 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:46.039353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:46.039354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:46.039358 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:46.039403 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:46.039417 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:46.039515 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:46.039570 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:11:46.039610 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":81,"threads_started":1}
I20260923 22:11:46.040958 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:46.041060 22814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } }
I20260923 22:11:46.041213 22814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:46.041242 22814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f65a6eb1a94451eb7b04bfcb3f0d835, State: Initialized, Role: FOLLOWER
I20260923 22:11:46.041353 22814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } }
I20260923 22:11:46.041440 22814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:46.041491 22814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:46.041560 22814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:46.042322 22814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } }
I20260923 22:11:46.042461 22814 leader_election.cc:302] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f65a6eb1a94451eb7b04bfcb3f0d835; no voters:
I20260923 22:11:46.042640 22814 leader_election.cc:288] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:46.042667 22824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:46.042809 22824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [term 1 LEADER]: Becoming Leader. State: Replica: 6f65a6eb1a94451eb7b04bfcb3f0d835, State: Running, Role: LEADER
I20260923 22:11:46.042906 22814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:46.042927 22824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } }
I20260923 22:11:46.043547 22824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f65a6eb1a94451eb7b04bfcb3f0d835. Latest consensus state: current_term: 1 leader_uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } } }
I20260923 22:11:46.043610 22825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f65a6eb1a94451eb7b04bfcb3f0d835" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } } }
I20260923 22:11:46.043730 22825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:46.043888 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:46.042521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60967 (local address 127.0.0.1:38081)
0923 22:11:46.042559 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:46.042561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:46.042718 (+ 157us) server_negotiation.cc:407] Connection header received
0923 22:11:46.042825 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:46.042826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:46.042831 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:46.042858 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:46.043487 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:46.043488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:46.043488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:46.043495 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:46.043508 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:46.043513 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:46.043704 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:46.043815 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 22:11:46.043829 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:11:46.044046 22824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:46.044268 22833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:46.044790 22833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:46.046281 22833 catalog_manager.cc:1347] Generated new cluster ID: fcffc51612c5481aab7771f40eb0e063
I20260923 22:11:46.046345 22833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:46.056057 22833 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:46.056285 22833 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:46.063616 22833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f65a6eb1a94451eb7b04bfcb3f0d835: Generated new TSK 0
I20260923 22:11:46.063817 22833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:46.068500 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 22746
2026-09-23T22:11:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:46 dist-test-slave-b5jz krb5kdc[22845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:46 dist-test-slave-b5jz krb5kdc[22845](info): set up 2 sockets
Sep 23 22:11:46 dist-test-slave-b5jz krb5kdc[22845](info): commencing operation
krb5kdc: starting...
W20260923 22:11:47.131695 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:47 dist-test-slave-b5jz krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:47.250159 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:47.330130 22858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:47.330384 22858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:47.330420 22858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:47.331640 22858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:47.331718 22858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:47.331743 22858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:47.331768 22858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:47.331794 22858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:47.333276 22858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:47.333550 22858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:47.333774 22858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:47.338598 22864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:47.338600 22863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:47.338603 22866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:47.338953 22858 server_base.cc:1034] running on GCE node
I20260923 22:11:47.339181 22858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:47.339416 22858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:47.340560 22858 hybrid_clock.cc:648] HybridClock initialized: now 1790201507340540 us; error 41 us; skew 500 ppm
Sep 23 22:11:47 dist-test-slave-b5jz krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201507, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:47.342959 22858 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:47.343326 22858 webserver.cc:460] Webserver started at http://127.0.0.1:45529/ using document root <none> and password file <none>
I20260923 22:11:47.343500 22858 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:47.343590 22858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:47.343716 22858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:47.344729 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "1c0a3ee0ec8d4af3b48f9357900529d3"
format_stamp: "Formatted at 2026-09-23 22:11:47 on dist-test-slave-b5jz"
server_key: "fa361f979da4cda0f4ea0fff02aaa71c"
server_key_iv: "998aebcc3415f8f1b3803c0026009b9b"
server_key_version: "encryptionkey@0"
I20260923 22:11:47.345024 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "1c0a3ee0ec8d4af3b48f9357900529d3"
format_stamp: "Formatted at 2026-09-23 22:11:47 on dist-test-slave-b5jz"
server_key: "fa361f979da4cda0f4ea0fff02aaa71c"
server_key_iv: "998aebcc3415f8f1b3803c0026009b9b"
server_key_version: "encryptionkey@0"
I20260923 22:11:47.346235 22858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:47.346887 22873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:47.347056 22858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:47.347154 22858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "1c0a3ee0ec8d4af3b48f9357900529d3"
format_stamp: "Formatted at 2026-09-23 22:11:47 on dist-test-slave-b5jz"
server_key: "fa361f979da4cda0f4ea0fff02aaa71c"
server_key_iv: "998aebcc3415f8f1b3803c0026009b9b"
server_key_version: "encryptionkey@0"
I20260923 22:11:47.347234 22858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:47.359426 22858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:47.361138 22858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:47.361299 22858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:47.365315 22858 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39205
I20260923 22:11:47.365345 22925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39205 every 8 connection(s)
I20260923 22:11:47.365684 22858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:47.366644 22926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:47.368857 22926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3: Bootstrap starting.
I20260923 22:11:47.369364 22926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:47.369601 22926 log.cc:824] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:47.370245 22926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3: No bootstrap required, opened a new log
I20260923 22:11:47.371837 22926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0a3ee0ec8d4af3b48f9357900529d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39205 } }
I20260923 22:11:47.371973 22926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:47.371996 22926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c0a3ee0ec8d4af3b48f9357900529d3, State: Initialized, Role: FOLLOWER
I20260923 22:11:47.372128 22926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0a3ee0ec8d4af3b48f9357900529d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39205 } }
I20260923 22:11:47.372202 22926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:47.372224 22926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:47.372252 22926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:47.372722 22926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0a3ee0ec8d4af3b48f9357900529d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39205 } }
I20260923 22:11:47.372807 22926 leader_election.cc:302] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c0a3ee0ec8d4af3b48f9357900529d3; no voters:
I20260923 22:11:47.372928 22926 leader_election.cc:288] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:47.373004 22929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:47.373183 22929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [term 1 LEADER]: Becoming Leader. State: Replica: 1c0a3ee0ec8d4af3b48f9357900529d3, State: Running, Role: LEADER
I20260923 22:11:47.373209 22926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:47.373283 22929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0a3ee0ec8d4af3b48f9357900529d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39205 } }
I20260923 22:11:47.373842 22930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c0a3ee0ec8d4af3b48f9357900529d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0a3ee0ec8d4af3b48f9357900529d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39205 } } }
I20260923 22:11:47.373952 22930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:47.374061 22931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c0a3ee0ec8d4af3b48f9357900529d3. Latest consensus state: current_term: 1 leader_uuid: "1c0a3ee0ec8d4af3b48f9357900529d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0a3ee0ec8d4af3b48f9357900529d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39205 } } }
I20260923 22:11:47.374223 22931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:47.374321 22939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:47.374776 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 22858
I20260923 22:11:47.374840 22939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:47.374905 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:47.375113 2003 external_mini_cluster.cc:1428] Setting key d01c35bdb78ee78adec025d528808d36
I20260923 22:11:47.376473 22939 catalog_manager.cc:1347] Generated new cluster ID: 54e98a443c154507bc0fad0065ef7f7b
I20260923 22:11:47.376533 22939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:47 dist-test-slave-b5jz krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:11:47.387156 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:47.375629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:54405 (local address 127.0.0.1:39205)
0923 22:11:47.375950 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:47.375958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:47.376152 (+ 194us) server_negotiation.cc:407] Connection header received
0923 22:11:47.376365 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:47.376375 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:47.376435 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:47.376582 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:47.377433 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:47.378854 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:47.379971 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:47.380569 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:47.380667 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:47.383463 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:47.383468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:47.383469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:47.383480 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:47.385466 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:47.385910 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:47.385914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:47.385914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:47.386021 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:47.386248 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:47.386249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:47.386249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:47.386553 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:47.386718 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:47.386812 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 22:11:47.386923 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":68,"threads_started":1}
I20260923 22:11:47.388130 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:47.395519 22939 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:47.396602 22939 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:47.398823 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:47.389575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:49591 (local address 127.0.0.1:39205)
0923 22:11:47.389610 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:47.389612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:47.391726 (+ 2114us) server_negotiation.cc:407] Connection header received
0923 22:11:47.391756 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:47.391757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:47.391768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:47.391836 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:47.392381 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:47.393165 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:47.394141 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:47.394469 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:47.394543 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:47.396047 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:47.396048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:47.396048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:47.396055 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:47.397596 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:47.398012 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:47.398013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:47.398013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:47.398075 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:47.398300 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:47.398301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:47.398301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:47.398426 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:47.398529 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:47.398674 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 22:11:47.398736 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:11:47.402285 22939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c0a3ee0ec8d4af3b48f9357900529d3: Generated new TSK 0
I20260923 22:11:47.402479 22939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:47.404440 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 22858
W20260923 22:11:47.408444 22843 connection.cc:535] client connection to 127.0.0.1:39205 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39205): Connection reset by peer (error 104)
2026-09-23T22:11:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:47 dist-test-slave-b5jz krb5kdc[22957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:47 dist-test-slave-b5jz krb5kdc[22957](info): set up 2 sockets
Sep 23 22:11:47 dist-test-slave-b5jz krb5kdc[22957](info): commencing operation
krb5kdc: starting...
W20260923 22:11:48.439740 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:48 dist-test-slave-b5jz krb5kdc[22957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:48.556167 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40021
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:48.635227 22970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:48.635437 22970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:48.635459 22970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:48.636643 22970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:48.636687 22970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:48.636703 22970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:48.636715 22970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:48.636727 22970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:48.638200 22970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:48.638453 22970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:48.638688 22970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:48.643520 22978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:48.643579 22975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:48.643491 22976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:48.643882 22970 server_base.cc:1034] running on GCE node
I20260923 22:11:48.644176 22970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:48.644469 22970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:48.645632 22970 hybrid_clock.cc:648] HybridClock initialized: now 1790201508645637 us; error 49 us; skew 500 ppm
Sep 23 22:11:48 dist-test-slave-b5jz krb5kdc[22957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201508, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:48.648144 22970 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:48.648622 22970 webserver.cc:460] Webserver started at http://127.0.0.1:37563/ using document root <none> and password file <none>
I20260923 22:11:48.648861 22970 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:48.648936 22970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:48.649076 22970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:48.650146 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "3b3b31f87b1a43b38d72c3e49d38e360"
format_stamp: "Formatted at 2026-09-23 22:11:48 on dist-test-slave-b5jz"
server_key: "18f7083556df09636ca5b1521f7f55ac"
server_key_iv: "81eca1640230ddeee1b4bda7621aa6d8"
server_key_version: "encryptionkey@0"
I20260923 22:11:48.650516 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "3b3b31f87b1a43b38d72c3e49d38e360"
format_stamp: "Formatted at 2026-09-23 22:11:48 on dist-test-slave-b5jz"
server_key: "18f7083556df09636ca5b1521f7f55ac"
server_key_iv: "81eca1640230ddeee1b4bda7621aa6d8"
server_key_version: "encryptionkey@0"
I20260923 22:11:48.651883 22970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:11:48.652680 22985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:48.652899 22970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:48.653008 22970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "3b3b31f87b1a43b38d72c3e49d38e360"
format_stamp: "Formatted at 2026-09-23 22:11:48 on dist-test-slave-b5jz"
server_key: "18f7083556df09636ca5b1521f7f55ac"
server_key_iv: "81eca1640230ddeee1b4bda7621aa6d8"
server_key_version: "encryptionkey@0"
I20260923 22:11:48.653097 22970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:48.664043 22970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:48.665769 22970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:48.665930 22970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:48.669996 22970 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40021
I20260923 22:11:48.670035 23037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40021 every 8 connection(s)
I20260923 22:11:48.670368 22970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:48.671365 23038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:48.673738 23038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360: Bootstrap starting.
I20260923 22:11:48.674299 23038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:48.674542 23038 log.cc:824] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:48.675186 23038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360: No bootstrap required, opened a new log
I20260923 22:11:48.676923 23038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b31f87b1a43b38d72c3e49d38e360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40021 } }
I20260923 22:11:48.677066 23038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:48.677091 23038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3b31f87b1a43b38d72c3e49d38e360, State: Initialized, Role: FOLLOWER
I20260923 22:11:48.677193 23038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b31f87b1a43b38d72c3e49d38e360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40021 } }
I20260923 22:11:48.677258 23038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:48.677309 23038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:48.677367 23038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:48.677924 23038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b31f87b1a43b38d72c3e49d38e360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40021 } }
I20260923 22:11:48.678041 23038 leader_election.cc:302] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b3b31f87b1a43b38d72c3e49d38e360; no voters:
I20260923 22:11:48.678232 23038 leader_election.cc:288] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:48.678272 23041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:48.678464 23041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [term 1 LEADER]: Becoming Leader. State: Replica: 3b3b31f87b1a43b38d72c3e49d38e360, State: Running, Role: LEADER
I20260923 22:11:48.678512 23038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:48.678565 23041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b31f87b1a43b38d72c3e49d38e360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40021 } }
I20260923 22:11:48.678982 23042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b3b31f87b1a43b38d72c3e49d38e360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b31f87b1a43b38d72c3e49d38e360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40021 } } }
I20260923 22:11:48.679006 23043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b3b31f87b1a43b38d72c3e49d38e360. Latest consensus state: current_term: 1 leader_uuid: "3b3b31f87b1a43b38d72c3e49d38e360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b31f87b1a43b38d72c3e49d38e360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40021 } } }
I20260923 22:11:48.679155 23042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:48.679179 23043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:48.679567 23046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:48.680160 23046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:48.680496 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 22970
I20260923 22:11:48.680612 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:48.680799 2003 external_mini_cluster.cc:1428] Setting key 32dd221f7cf52349468f9b7835557f86
I20260923 22:11:48.682065 23046 catalog_manager.cc:1347] Generated new cluster ID: c41dc5e77e1a4bfbbab4be225f484881
I20260923 22:11:48.682122 23046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:48.687062 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:48.681206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:33205 (local address 127.0.0.1:40021)
0923 22:11:48.681382 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:48.681391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:48.682401 (+ 1010us) server_negotiation.cc:407] Connection header received
0923 22:11:48.682566 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:48.682573 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:48.682621 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:48.682771 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:48.683016 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:48.684137 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:48.685022 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:48.685486 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:48.685629 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:48.686421 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:48.686425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:48.686426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:48.686616 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:48.686647 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:48.686659 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:48.686768 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:48.686854 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 22:11:48.686885 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260923 22:11:48.688019 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:48.691023 23046 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:48.691730 23046 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:48.697640 23046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b3b31f87b1a43b38d72c3e49d38e360: Generated new TSK 0
I20260923 22:11:48.697814 23046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:48.698521 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:48.689529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:37679 (local address 127.0.0.1:40021)
0923 22:11:48.689571 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:48.689573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:48.690678 (+ 1105us) server_negotiation.cc:407] Connection header received
0923 22:11:48.690724 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:48.690725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:48.690737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:48.690816 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:48.695736 (+ 4920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:48.696285 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:48.697069 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:48.697358 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:48.697437 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:48.698233 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:48.698234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:48.698234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:48.698239 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:48.698261 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:48.698266 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:48.698347 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:48.698421 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:11:48.698435 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:11:48.700652 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 22970
W20260923 22:11:48.705205 22955 connection.cc:535] client connection to 127.0.0.1:40021 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40021): Connection reset by peer (error 104)
2026-09-23T22:11:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:48 dist-test-slave-b5jz krb5kdc[23069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:48 dist-test-slave-b5jz krb5kdc[23069](info): set up 2 sockets
Sep 23 22:11:48 dist-test-slave-b5jz krb5kdc[23069](info): commencing operation
krb5kdc: starting...
W20260923 22:11:49.728142 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:49 dist-test-slave-b5jz krb5kdc[23069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:49.847185 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:49.930085 23082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:49.930347 23082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:49.930387 23082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:49.931658 23082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:49.931718 23082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:49.931741 23082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:49.931773 23082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:49.931802 23082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:49.933316 23082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:49.933594 23082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:49.933831 23082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:49.938799 23090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:49.938820 23088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:49.939009 23082 server_base.cc:1034] running on GCE node
W20260923 22:11:49.938854 23087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:49.939292 23082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:49.939685 23082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:49.940828 23082 hybrid_clock.cc:648] HybridClock initialized: now 1790201509940820 us; error 29 us; skew 500 ppm
Sep 23 22:11:49 dist-test-slave-b5jz krb5kdc[23069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201509, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:49.943529 23082 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:49.944044 23082 webserver.cc:460] Webserver started at http://127.0.0.1:38685/ using document root <none> and password file <none>
I20260923 22:11:49.944283 23082 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:49.944352 23082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:49.944510 23082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:49.945616 23082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "8a995523e18240709959d8a591de5e46"
format_stamp: "Formatted at 2026-09-23 22:11:49 on dist-test-slave-b5jz"
server_key: "7a941f3a940e04a77948fc625d5a893b"
server_key_iv: "fb593e53d7932f4f88cda0962ea60ff1"
server_key_version: "encryptionkey@0"
I20260923 22:11:49.945994 23082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "8a995523e18240709959d8a591de5e46"
format_stamp: "Formatted at 2026-09-23 22:11:49 on dist-test-slave-b5jz"
server_key: "7a941f3a940e04a77948fc625d5a893b"
server_key_iv: "fb593e53d7932f4f88cda0962ea60ff1"
server_key_version: "encryptionkey@0"
I20260923 22:11:49.947376 23082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:11:49.948225 23097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:49.948414 23082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:49.948526 23082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "8a995523e18240709959d8a591de5e46"
format_stamp: "Formatted at 2026-09-23 22:11:49 on dist-test-slave-b5jz"
server_key: "7a941f3a940e04a77948fc625d5a893b"
server_key_iv: "fb593e53d7932f4f88cda0962ea60ff1"
server_key_version: "encryptionkey@0"
I20260923 22:11:49.948613 23082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:49.952119 23082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:49.954160 23082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:49.954319 23082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:49.958503 23082 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38295
I20260923 22:11:49.958537 23149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38295 every 8 connection(s)
I20260923 22:11:49.958870 23082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:49.959965 23150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:49.961804 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 23082
I20260923 22:11:49.961905 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:49.962124 2003 external_mini_cluster.cc:1428] Setting key 50be3510be242e8d5362d6487770a311
I20260923 22:11:49.962512 23150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46: Bootstrap starting.
I20260923 22:11:49.963308 23150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:49.963652 23150 log.cc:824] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:49.964483 23150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46: No bootstrap required, opened a new log
I20260923 22:11:49.964977 23153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:49.962678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:34283 (local address 127.0.0.1:38295)
0923 22:11:49.962877 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:49.962887 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:49.962926 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:11:49.963530 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:49.963539 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:49.963622 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:49.963785 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:49.964538 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:49.964542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:49.964542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:49.964547 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:49.964593 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:49.964610 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:49.964691 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:49.964744 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 22:11:49.964787 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
I20260923 22:11:49.965989 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:49.966935 23150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a995523e18240709959d8a591de5e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260923 22:11:49.967123 23150 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:49.967218 23150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a995523e18240709959d8a591de5e46, State: Initialized, Role: FOLLOWER
I20260923 22:11:49.967394 23150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a995523e18240709959d8a591de5e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260923 22:11:49.967509 23150 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:49.967590 23150 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:49.967656 23150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:49.968439 23150 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a995523e18240709959d8a591de5e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260923 22:11:49.968554 23150 leader_election.cc:302] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a995523e18240709959d8a591de5e46; no voters:
I20260923 22:11:49.968750 23150 leader_election.cc:288] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:49.968977 23153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:49.967663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:59699 (local address 127.0.0.1:38295)
0923 22:11:49.967702 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:49.967704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:49.967979 (+ 275us) server_negotiation.cc:407] Connection header received
0923 22:11:49.968001 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:49.968002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:49.968006 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:49.968085 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:49.968704 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:49.968705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:49.968706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:49.968710 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:49.968725 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:49.968730 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:49.968795 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:49.968865 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:11:49.968878 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:11:49.969022 23150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:49.968868 23160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:49.969266 23160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [term 1 LEADER]: Becoming Leader. State: Replica: 8a995523e18240709959d8a591de5e46, State: Running, Role: LEADER
I20260923 22:11:49.969434 23160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a995523e18240709959d8a591de5e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260923 22:11:49.969841 23160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a995523e18240709959d8a591de5e46. Latest consensus state: current_term: 1 leader_uuid: "8a995523e18240709959d8a591de5e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a995523e18240709959d8a591de5e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } } }
I20260923 22:11:49.969933 23160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:49.969849 23162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a995523e18240709959d8a591de5e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a995523e18240709959d8a591de5e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } } }
I20260923 22:11:49.970111 23162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46 [sys.catalog]: This master's current role is: LEADER
W20260923 22:11:49.970870 23176 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:11:49.970913 23176 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:11:49.970974 23171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:49.971150 23171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:49.972635 23171 catalog_manager.cc:1347] Generated new cluster ID: 49989311737c40e49c8e056338e9992f
I20260923 22:11:49.972687 23171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:49.980544 23171 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:49.980722 23171 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:49.985186 23171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a995523e18240709959d8a591de5e46: Generated new TSK 0
I20260923 22:11:49.985378 23171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:49.987174 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 23082
2026-09-23T22:11:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:50 dist-test-slave-b5jz krb5kdc[23181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:50 dist-test-slave-b5jz krb5kdc[23181](info): set up 2 sockets
Sep 23 22:11:50 dist-test-slave-b5jz krb5kdc[23181](info): commencing operation
krb5kdc: starting...
W20260923 22:11:51.015719 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:51 dist-test-slave-b5jz krb5kdc[23181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:51.132288 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:51.213683 23194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:51.213894 23194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:51.213917 23194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:51.215096 23194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:51.215143 23194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:51.215158 23194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:51.215171 23194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:51.215183 23194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:51.216756 23194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42965
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:51.217011 23194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:51.217226 23194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:11:51.222183 23194 server_base.cc:1034] running on GCE node
W20260923 22:11:51.222203 23199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:51.222251 23200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:51.222234 23202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:51.222707 23194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:51.222980 23194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:51.224126 23194 hybrid_clock.cc:648] HybridClock initialized: now 1790201511224111 us; error 31 us; skew 500 ppm
Sep 23 22:11:51 dist-test-slave-b5jz krb5kdc[23181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201511, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:51.232772 23194 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:51.233254 23194 webserver.cc:460] Webserver started at http://127.0.0.1:35459/ using document root <none> and password file <none>
I20260923 22:11:51.233505 23194 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:51.233549 23194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:51.233707 23194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:51.234786 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "81a5052173584c2aa17c1f94b3ebaa81"
format_stamp: "Formatted at 2026-09-23 22:11:51 on dist-test-slave-b5jz"
server_key: "863c83fe3f24133de0d9955c4168a8cc"
server_key_iv: "6aff43514fa6f915b9d470cbaa70177b"
server_key_version: "encryptionkey@0"
I20260923 22:11:51.235220 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "81a5052173584c2aa17c1f94b3ebaa81"
format_stamp: "Formatted at 2026-09-23 22:11:51 on dist-test-slave-b5jz"
server_key: "863c83fe3f24133de0d9955c4168a8cc"
server_key_iv: "6aff43514fa6f915b9d470cbaa70177b"
server_key_version: "encryptionkey@0"
I20260923 22:11:51.236653 23194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:11:51.237394 23209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:51.237572 23194 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:51.237685 23194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "81a5052173584c2aa17c1f94b3ebaa81"
format_stamp: "Formatted at 2026-09-23 22:11:51 on dist-test-slave-b5jz"
server_key: "863c83fe3f24133de0d9955c4168a8cc"
server_key_iv: "6aff43514fa6f915b9d470cbaa70177b"
server_key_version: "encryptionkey@0"
I20260923 22:11:51.237761 23194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:51.259838 23194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:51.261756 23194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:51.261942 23194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:51.266001 23194 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42965
I20260923 22:11:51.266048 23261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42965 every 8 connection(s)
I20260923 22:11:51.266424 23194 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:51.266979 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 23194
I20260923 22:11:51.267112 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:51.267323 2003 external_mini_cluster.cc:1428] Setting key ac16a9d4150e3917caf3bf766b4282e6
I20260923 22:11:51.267491 23262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:51.270968 23262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81: Bootstrap starting.
I20260923 22:11:51.271672 23262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:51.271967 23262 log.cc:824] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:51.272634 23262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81: No bootstrap required, opened a new log
Sep 23 22:11:51 dist-test-slave-b5jz krb5kdc[23181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:11:51.274425 23262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a5052173584c2aa17c1f94b3ebaa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } }
I20260923 22:11:51.274595 23262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:51.274642 23262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a5052173584c2aa17c1f94b3ebaa81, State: Initialized, Role: FOLLOWER
I20260923 22:11:51.274775 23262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a5052173584c2aa17c1f94b3ebaa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } }
I20260923 22:11:51.274865 23262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:51.274900 23262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:51.274952 23262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:51.275720 23262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a5052173584c2aa17c1f94b3ebaa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } }
I20260923 22:11:51.275844 23262 leader_election.cc:302] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81a5052173584c2aa17c1f94b3ebaa81; no voters:
I20260923 22:11:51.276012 23262 leader_election.cc:288] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:51.276048 23267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:51.276160 23267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [term 1 LEADER]: Becoming Leader. State: Replica: 81a5052173584c2aa17c1f94b3ebaa81, State: Running, Role: LEADER
I20260923 22:11:51.276294 23267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a5052173584c2aa17c1f94b3ebaa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } }
I20260923 22:11:51.276273 23262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:51.276772 23268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81a5052173584c2aa17c1f94b3ebaa81. Latest consensus state: current_term: 1 leader_uuid: "81a5052173584c2aa17c1f94b3ebaa81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a5052173584c2aa17c1f94b3ebaa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } } }
I20260923 22:11:51.276764 23269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81a5052173584c2aa17c1f94b3ebaa81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a5052173584c2aa17c1f94b3ebaa81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } } }
I20260923 22:11:51.276865 23268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:51.276875 23269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:51.277074 23273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:51.277597 23273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:51.279053 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:51.267932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:58805 (local address 127.0.0.1:42965)
0923 22:11:51.268128 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:51.268138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:51.268172 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:11:51.268747 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:51.268756 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:51.268823 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:51.268991 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:51.269593 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:51.270941 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:51.271813 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:51.272278 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:51.272411 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:51.274810 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:51.274814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:51.274814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:51.274825 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:51.277205 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:51.277784 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:51.277787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:51.277788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:51.277974 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:51.278240 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:51.278240 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:51.278240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:51.278490 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:51.278669 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:51.278799 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 22:11:51.278898 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260923 22:11:51.279498 23273 catalog_manager.cc:1347] Generated new cluster ID: 352a797d16fd422d8d7ff002b8d27b98
I20260923 22:11:51.279587 23273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:51.280059 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:51.299350 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:51.281470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:60359 (local address 127.0.0.1:42965)
0923 22:11:51.281510 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:51.281511 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:51.291747 (+ 10236us) server_negotiation.cc:407] Connection header received
0923 22:11:51.291821 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:51.291822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:51.291834 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:51.291890 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:51.292593 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:51.293379 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:51.294423 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:51.294744 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:51.294807 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:51.296337 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:51.296338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:51.296339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:51.296346 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:51.297979 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:51.298421 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:51.298422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:51.298423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:51.298481 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:51.298722 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:51.298723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:51.298723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:51.298845 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:51.298969 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:51.299195 (+ 226us) server_negotiation.cc:299] Negotiation successful
0923 22:11:51.299261 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:11:51.299443 23273 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:51.300189 23273 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:51.304987 23273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81a5052173584c2aa17c1f94b3ebaa81: Generated new TSK 0
I20260923 22:11:51.305214 23273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:51.307760 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 23194
W20260923 22:11:51.312635 23179 connection.cc:535] client connection to 127.0.0.1:42965 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42965): Connection reset by peer (error 104)
2026-09-23T22:11:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:51 dist-test-slave-b5jz krb5kdc[23293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:51 dist-test-slave-b5jz krb5kdc[23293](info): set up 2 sockets
Sep 23 22:11:51 dist-test-slave-b5jz krb5kdc[23293](info): commencing operation
krb5kdc: starting...
W20260923 22:11:52.359571 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:52 dist-test-slave-b5jz krb5kdc[23293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:52.474742 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45275
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:52.554898 23306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:52.555146 23306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:52.555199 23306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:52.556406 23306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:52.556478 23306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:52.556502 23306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:52.556523 23306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:52.556581 23306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:52.558610 23306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45275
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:52.558885 23306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:52.559124 23306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:52.563855 23312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:52.563942 23311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:52.563860 23314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:52.564045 23306 server_base.cc:1034] running on GCE node
I20260923 22:11:52.564286 23306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:52.564594 23306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:52.565773 23306 hybrid_clock.cc:648] HybridClock initialized: now 1790201512565729 us; error 34 us; skew 500 ppm
Sep 23 22:11:52 dist-test-slave-b5jz krb5kdc[23293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201512, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:52.568403 23306 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:52.568964 23306 webserver.cc:460] Webserver started at http://127.0.0.1:39441/ using document root <none> and password file <none>
I20260923 22:11:52.569182 23306 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:52.569226 23306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:52.569370 23306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:52.570395 23306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "6214bc533d7645329234408992ad2c07"
format_stamp: "Formatted at 2026-09-23 22:11:52 on dist-test-slave-b5jz"
server_key: "8939b0b73c0c01f1f2dc549f9a7c7e40"
server_key_iv: "7ac8f2f2602f6d6207591eac4632762a"
server_key_version: "encryptionkey@0"
I20260923 22:11:52.570748 23306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "6214bc533d7645329234408992ad2c07"
format_stamp: "Formatted at 2026-09-23 22:11:52 on dist-test-slave-b5jz"
server_key: "8939b0b73c0c01f1f2dc549f9a7c7e40"
server_key_iv: "7ac8f2f2602f6d6207591eac4632762a"
server_key_version: "encryptionkey@0"
I20260923 22:11:52.572103 23306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:52.572825 23321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:52.572994 23306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:52.573063 23306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "6214bc533d7645329234408992ad2c07"
format_stamp: "Formatted at 2026-09-23 22:11:52 on dist-test-slave-b5jz"
server_key: "8939b0b73c0c01f1f2dc549f9a7c7e40"
server_key_iv: "7ac8f2f2602f6d6207591eac4632762a"
server_key_version: "encryptionkey@0"
I20260923 22:11:52.573148 23306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:52.587322 23306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:52.589174 23306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:52.589334 23306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:52.593065 23306 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45275
I20260923 22:11:52.593106 23373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45275 every 8 connection(s)
I20260923 22:11:52.593472 23306 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:52.594331 23374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:52.596674 23374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07: Bootstrap starting.
I20260923 22:11:52.597265 23374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:52.597551 23374 log.cc:824] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:52.598235 23374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07: No bootstrap required, opened a new log
I20260923 22:11:52.599179 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 23306
I20260923 22:11:52.599298 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:52.599501 2003 external_mini_cluster.cc:1428] Setting key a3139a9d16262bdbd8f67eb5b056546a
I20260923 22:11:52.600075 23374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6214bc533d7645329234408992ad2c07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } }
I20260923 22:11:52.600271 23374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:52.600319 23374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6214bc533d7645329234408992ad2c07, State: Initialized, Role: FOLLOWER
I20260923 22:11:52.600455 23374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6214bc533d7645329234408992ad2c07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } }
I20260923 22:11:52.600536 23374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:52.600594 23374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:52.600646 23374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:52.601161 23374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6214bc533d7645329234408992ad2c07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } }
I20260923 22:11:52.601281 23374 leader_election.cc:302] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6214bc533d7645329234408992ad2c07; no voters:
I20260923 22:11:52.601473 23374 leader_election.cc:288] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:52.601507 23379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:52.601622 23379 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [term 1 LEADER]: Becoming Leader. State: Replica: 6214bc533d7645329234408992ad2c07, State: Running, Role: LEADER
I20260923 22:11:52.601732 23374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:52.601759 23379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6214bc533d7645329234408992ad2c07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } }
I20260923 22:11:52.602165 23379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6214bc533d7645329234408992ad2c07. Latest consensus state: current_term: 1 leader_uuid: "6214bc533d7645329234408992ad2c07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6214bc533d7645329234408992ad2c07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } } }
I20260923 22:11:52.602417 23379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:52.602332 23380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6214bc533d7645329234408992ad2c07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6214bc533d7645329234408992ad2c07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } } }
I20260923 22:11:52.602875 23380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:52.603017 23388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:52.603596 23388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:52.605233 23388 catalog_manager.cc:1347] Generated new cluster ID: 8210fb2c079d4250a923bc2ae0d23620
I20260923 22:11:52.605280 23388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:52.605595 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:52.600028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:55905 (local address 127.0.0.1:45275)
0923 22:11:52.600226 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:52.600234 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:52.600258 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:11:52.600381 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:52.600389 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:52.600448 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:52.600600 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:52.600795 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:52.602258 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:52.603419 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:52.604018 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:52.604439 (+ 421us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:52.604886 (+ 447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:52.604890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:52.604891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:52.605085 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:52.605155 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:52.605173 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:52.605269 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:52.605356 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:11:52.605383 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260923 22:11:52.606504 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:52.611207 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:52.607959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:52269 (local address 127.0.0.1:45275)
0923 22:11:52.607989 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:52.607991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:52.608093 (+ 102us) server_negotiation.cc:407] Connection header received
0923 22:11:52.608167 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:52.608167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:52.608176 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:52.608200 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:52.608388 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:52.609062 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:52.609975 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:52.610265 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:52.610296 (+ 31us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:52.610978 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:52.610979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:52.610979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:52.610983 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:52.610994 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:52.610997 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:52.611024 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:52.611132 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 22:11:52.611141 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:11:52.626425 23388 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:52.627139 23388 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:52.636008 23388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6214bc533d7645329234408992ad2c07: Generated new TSK 0
I20260923 22:11:52.636184 23388 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:52.639991 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 23306
W20260923 22:11:52.644332 23291 connection.cc:535] client connection to 127.0.0.1:45275 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45275): Connection reset by peer (error 104)
2026-09-23T22:11:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:52 dist-test-slave-b5jz krb5kdc[23405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:52 dist-test-slave-b5jz krb5kdc[23405](info): set up 2 sockets
Sep 23 22:11:52 dist-test-slave-b5jz krb5kdc[23405](info): commencing operation
krb5kdc: starting...
W20260923 22:11:53.667745 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:53 dist-test-slave-b5jz krb5kdc[23405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:53.784466 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:53.863287 23418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:53.863473 23418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:53.863495 23418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:53.864667 23418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:53.864710 23418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:53.864725 23418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:53.864737 23418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:53.864750 23418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:53.866210 23418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:53.866425 23418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:53.866621 23418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:53.871186 23423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:53.871227 23424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:53.871376 23418 server_base.cc:1034] running on GCE node
W20260923 22:11:53.871251 23426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:53.871759 23418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:53.872104 23418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:53.873291 23418 hybrid_clock.cc:648] HybridClock initialized: now 1790201513873275 us; error 27 us; skew 500 ppm
Sep 23 22:11:53 dist-test-slave-b5jz krb5kdc[23405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201513, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:53.875797 23418 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:53.876277 23418 webserver.cc:460] Webserver started at http://127.0.0.1:44887/ using document root <none> and password file <none>
I20260923 22:11:53.876473 23418 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:53.876513 23418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:53.876595 23418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:53.877535 23418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "4de564d429224bb3994056a5ffe0d49f"
format_stamp: "Formatted at 2026-09-23 22:11:53 on dist-test-slave-b5jz"
server_key: "14d51e56f8bcf5fbf9296a6a7ef260c7"
server_key_iv: "7855f9b7ea0b9bc90f98b2e137be3035"
server_key_version: "encryptionkey@0"
I20260923 22:11:53.877817 23418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "4de564d429224bb3994056a5ffe0d49f"
format_stamp: "Formatted at 2026-09-23 22:11:53 on dist-test-slave-b5jz"
server_key: "14d51e56f8bcf5fbf9296a6a7ef260c7"
server_key_iv: "7855f9b7ea0b9bc90f98b2e137be3035"
server_key_version: "encryptionkey@0"
I20260923 22:11:53.878985 23418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:11:53.879736 23433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:53.879885 23418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:53.879979 23418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "4de564d429224bb3994056a5ffe0d49f"
format_stamp: "Formatted at 2026-09-23 22:11:53 on dist-test-slave-b5jz"
server_key: "14d51e56f8bcf5fbf9296a6a7ef260c7"
server_key_iv: "7855f9b7ea0b9bc90f98b2e137be3035"
server_key_version: "encryptionkey@0"
I20260923 22:11:53.880033 23418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:53.883188 23418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:53.885438 23418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:53.885557 23418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:53.889261 23418 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44741
I20260923 22:11:53.889324 23485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44741 every 8 connection(s)
I20260923 22:11:53.889635 23418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:53.890605 23486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:53.892856 23486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f: Bootstrap starting.
I20260923 22:11:53.893500 23486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:53.893777 23486 log.cc:824] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:53.894423 23486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f: No bootstrap required, opened a new log
I20260923 22:11:53.896034 23486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de564d429224bb3994056a5ffe0d49f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44741 } }
I20260923 22:11:53.896198 23486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:53.896245 23486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de564d429224bb3994056a5ffe0d49f, State: Initialized, Role: FOLLOWER
I20260923 22:11:53.896407 23486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de564d429224bb3994056a5ffe0d49f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44741 } }
I20260923 22:11:53.896492 23486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:53.896533 23486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:53.896584 23486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:53.897111 23486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de564d429224bb3994056a5ffe0d49f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44741 } }
I20260923 22:11:53.897228 23486 leader_election.cc:302] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4de564d429224bb3994056a5ffe0d49f; no voters:
I20260923 22:11:53.897413 23486 leader_election.cc:288] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:53.897461 23489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:53.897660 23486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:53.897668 23489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [term 1 LEADER]: Becoming Leader. State: Replica: 4de564d429224bb3994056a5ffe0d49f, State: Running, Role: LEADER
I20260923 22:11:53.897771 23489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de564d429224bb3994056a5ffe0d49f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44741 } }
I20260923 22:11:53.898136 23490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4de564d429224bb3994056a5ffe0d49f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de564d429224bb3994056a5ffe0d49f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44741 } } }
I20260923 22:11:53.898228 23490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:53.898141 23491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4de564d429224bb3994056a5ffe0d49f. Latest consensus state: current_term: 1 leader_uuid: "4de564d429224bb3994056a5ffe0d49f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de564d429224bb3994056a5ffe0d49f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44741 } } }
I20260923 22:11:53.898357 23491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:53.898536 23493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:53.898732 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 23418
I20260923 22:11:53.898864 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:53.899063 23493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:53.899061 2003 external_mini_cluster.cc:1428] Setting key 3eff347cd296dfd1d303404054d84aed
I20260923 22:11:53.901260 23493 catalog_manager.cc:1347] Generated new cluster ID: ea9acb0fb59342aca40cd6090ade9c99
I20260923 22:11:53.901314 23493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:53.901754 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:53.899688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:53479 (local address 127.0.0.1:44741)
0923 22:11:53.900061 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:53.900070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:53.900099 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:11:53.900208 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:53.900215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:53.900414 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:53.900576 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:53.901268 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:53.901271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:53.901272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:53.901277 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:53.901321 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:53.901337 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:53.901376 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:53.901515 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 22:11:53.901566 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":60,"threads_started":1}
I20260923 22:11:53.902652 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:53.908996 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:53.907825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.123:51009 (local address 127.0.0.1:44741)
0923 22:11:53.907863 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:53.907864 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:53.908135 (+ 271us) server_negotiation.cc:407] Connection header received
0923 22:11:53.908160 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:53.908161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:53.908183 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:53.908236 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:53.908791 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:53.908793 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:53.908793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:53.908797 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:53.908812 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:53.908816 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:53.908875 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:53.908924 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 22:11:53.908937 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:11:53.917084 23493 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:53.917249 23493 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:53.924561 23493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4de564d429224bb3994056a5ffe0d49f: Generated new TSK 0
I20260923 22:11:53.924731 23493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:53.931764 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 23418
2026-09-23T22:11:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:53 dist-test-slave-b5jz krb5kdc[23517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:53 dist-test-slave-b5jz krb5kdc[23517](info): set up 2 sockets
Sep 23 22:11:53 dist-test-slave-b5jz krb5kdc[23517](info): commencing operation
krb5kdc: starting...
W20260923 22:11:54.983705 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:55 dist-test-slave-b5jz krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:11:55.102608 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:44257
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:44257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:55.183782 23530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:55.183987 23530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:55.184011 23530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:55.185117 23530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:55.185159 23530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:55.185173 23530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:55.185185 23530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:55.185197 23530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:55.186600 23530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:44257
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:44257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:55.186795 23530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:55.186971 23530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:55.191706 23538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:55.191679 23536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:55.191856 23530 server_base.cc:1034] running on GCE node
W20260923 22:11:55.191915 23535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:55.192160 23530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:55.192474 23530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:55.193615 23530 hybrid_clock.cc:648] HybridClock initialized: now 1790201515193595 us; error 36 us; skew 500 ppm
Sep 23 22:11:55 dist-test-slave-b5jz krb5kdc[23517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:55.196070 23530 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:11:55.196538 23530 webserver.cc:460] Webserver started at http://127.1.244.254:37891/ using document root <none> and password file <none>
I20260923 22:11:55.196772 23530 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:55.196841 23530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:55.196993 23530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:55.198010 23530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "7969626b0e934f4ba1d45273fea84dcc"
format_stamp: "Formatted at 2026-09-23 22:11:55 on dist-test-slave-b5jz"
server_key: "3dc233730aa40e5dd383d3bacbafaf4d"
server_key_iv: "cfed173592bb0b35a09f57735b8f1209"
server_key_version: "encryptionkey@0"
I20260923 22:11:55.198374 23530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "7969626b0e934f4ba1d45273fea84dcc"
format_stamp: "Formatted at 2026-09-23 22:11:55 on dist-test-slave-b5jz"
server_key: "3dc233730aa40e5dd383d3bacbafaf4d"
server_key_iv: "cfed173592bb0b35a09f57735b8f1209"
server_key_version: "encryptionkey@0"
I20260923 22:11:55.199682 23530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:11:55.200380 23545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:55.200546 23530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:55.200654 23530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "7969626b0e934f4ba1d45273fea84dcc"
format_stamp: "Formatted at 2026-09-23 22:11:55 on dist-test-slave-b5jz"
server_key: "3dc233730aa40e5dd383d3bacbafaf4d"
server_key_iv: "cfed173592bb0b35a09f57735b8f1209"
server_key_version: "encryptionkey@0"
I20260923 22:11:55.200742 23530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:55.210680 23530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:55.212925 23530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:55.213085 23530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:55.217111 23530 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:44257
I20260923 22:11:55.217164 23597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:44257 every 8 connection(s)
I20260923 22:11:55.217501 23530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:55.218362 23598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:55.220472 23598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc: Bootstrap starting.
I20260923 22:11:55.221020 23598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:55.221305 23598 log.cc:824] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:55.221951 23598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc: No bootstrap required, opened a new log
I20260923 22:11:55.223459 23598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969626b0e934f4ba1d45273fea84dcc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44257 } }
I20260923 22:11:55.223649 23598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:55.223676 23598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7969626b0e934f4ba1d45273fea84dcc, State: Initialized, Role: FOLLOWER
I20260923 22:11:55.223843 23598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969626b0e934f4ba1d45273fea84dcc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44257 } }
I20260923 22:11:55.223927 23598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:55.223979 23598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:55.224040 23598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:55.224843 23598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969626b0e934f4ba1d45273fea84dcc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44257 } }
I20260923 22:11:55.224951 23598 leader_election.cc:302] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7969626b0e934f4ba1d45273fea84dcc; no voters:
I20260923 22:11:55.225136 23598 leader_election.cc:288] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:55.225184 23601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:55.225365 23601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [term 1 LEADER]: Becoming Leader. State: Replica: 7969626b0e934f4ba1d45273fea84dcc, State: Running, Role: LEADER
I20260923 22:11:55.225389 23598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:55.225517 23601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969626b0e934f4ba1d45273fea84dcc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44257 } }
I20260923 22:11:55.225898 23602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7969626b0e934f4ba1d45273fea84dcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969626b0e934f4ba1d45273fea84dcc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44257 } } }
I20260923 22:11:55.226004 23602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:55.225899 23603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7969626b0e934f4ba1d45273fea84dcc. Latest consensus state: current_term: 1 leader_uuid: "7969626b0e934f4ba1d45273fea84dcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969626b0e934f4ba1d45273fea84dcc" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44257 } } }
I20260923 22:11:55.226099 23603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:55.226270 23610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:55.226743 23610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:55.227118 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 23530
I20260923 22:11:55.227248 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:55.227427 2003 external_mini_cluster.cc:1428] Setting key 17e81959208e2477f9a9f990e1858567
I20260923 22:11:55.228487 23610 catalog_manager.cc:1347] Generated new cluster ID: 8f5261d53ae3443dbc2ad25bbaf12bd4
I20260923 22:11:55.228536 23610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:55 dist-test-slave-b5jz krb5kdc[23517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:55.250631 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:55.227877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49773 (local address 127.1.244.254:44257)
0923 22:11:55.228048 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:55.228057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:55.228096 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:11:55.228222 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:55.228229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:55.228284 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:55.228431 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:55.240297 (+ 11866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:55.241846 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:55.242928 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:55.243686 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:55.243815 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:55.246740 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:55.246745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:55.246745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:55.246759 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:55.248786 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:55.249270 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:55.249273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:55.249274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:55.249386 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:55.249652 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:55.249653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:55.249653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:55.249967 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:55.250109 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:55.250279 (+ 170us) server_negotiation.cc:299] Negotiation successful
0923 22:11:55.250384 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260923 22:11:55.251783 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:55.260542 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:55.253201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47791 (local address 127.1.244.254:44257)
0923 22:11:55.253234 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:55.253236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:55.253373 (+ 137us) server_negotiation.cc:407] Connection header received
0923 22:11:55.253442 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:55.253443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:55.253452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:55.253506 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:55.254122 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:55.254857 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:55.255957 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:55.256269 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:55.256332 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:55.257751 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:55.257752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:55.257753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:55.257759 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:55.259284 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:55.259737 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:55.259738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:55.259738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:55.259794 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:55.260036 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:55.260037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:55.260037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:55.260154 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:55.260264 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:55.260393 (+ 129us) server_negotiation.cc:299] Negotiation successful
0923 22:11:55.260453 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:11:55.262910 23610 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:55.263693 23610 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:55.276084 23610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7969626b0e934f4ba1d45273fea84dcc: Generated new TSK 0
I20260923 22:11:55.276309 23610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:55.279247 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 23530
W20260923 22:11:55.284065 23515 connection.cc:535] client connection to 127.1.244.254:44257 recv error: Network error: failed to read from TLS socket (remote: 127.1.244.254:44257): Connection reset by peer (error 104)
2026-09-23T22:11:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:55 dist-test-slave-b5jz krb5kdc[23629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:55 dist-test-slave-b5jz krb5kdc[23629](info): set up 2 sockets
Sep 23 22:11:55 dist-test-slave-b5jz krb5kdc[23629](info): commencing operation
krb5kdc: starting...
W20260923 22:11:56.371776 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:56 dist-test-slave-b5jz krb5kdc[23629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:11:56.486387 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35065
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:56.567358 23642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:56.567643 23642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:56.567715 23642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:56.568917 23642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:56.568985 23642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:56.569011 23642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:56.569042 23642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:56.569073 23642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:56.570556 23642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35065
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:56.570827 23642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:56.571049 23642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:56.575861 23648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:56.575980 23642 server_base.cc:1034] running on GCE node
W20260923 22:11:56.575894 23650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:56.575861 23647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:56.576341 23642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:56.576674 23642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:56.577849 23642 hybrid_clock.cc:648] HybridClock initialized: now 1790201516577832 us; error 45 us; skew 500 ppm
Sep 23 22:11:56 dist-test-slave-b5jz krb5kdc[23629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:56.586063 23642 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:11:56.586565 23642 webserver.cc:460] Webserver started at http://127.1.244.254:46249/ using document root <none> and password file <none>
I20260923 22:11:56.586827 23642 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:56.586903 23642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:56.587056 23642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:56.588227 23642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "7f78fb2f9944435f95ba5a3f8452a431"
format_stamp: "Formatted at 2026-09-23 22:11:56 on dist-test-slave-b5jz"
server_key: "c0d20a2248db9858db816ab48758e92b"
server_key_iv: "4070a803ffab6a50871ffef56aac2ed8"
server_key_version: "encryptionkey@0"
I20260923 22:11:56.588598 23642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "7f78fb2f9944435f95ba5a3f8452a431"
format_stamp: "Formatted at 2026-09-23 22:11:56 on dist-test-slave-b5jz"
server_key: "c0d20a2248db9858db816ab48758e92b"
server_key_iv: "4070a803ffab6a50871ffef56aac2ed8"
server_key_version: "encryptionkey@0"
I20260923 22:11:56.589953 23642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:56.590723 23657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:56.590900 23642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:56.591013 23642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "7f78fb2f9944435f95ba5a3f8452a431"
format_stamp: "Formatted at 2026-09-23 22:11:56 on dist-test-slave-b5jz"
server_key: "c0d20a2248db9858db816ab48758e92b"
server_key_iv: "4070a803ffab6a50871ffef56aac2ed8"
server_key_version: "encryptionkey@0"
I20260923 22:11:56.591090 23642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:56.624320 23642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:56.626880 23642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:56.627061 23642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:56.631321 23642 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35065
I20260923 22:11:56.631352 23709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35065 every 8 connection(s)
I20260923 22:11:56.631733 23642 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:56.632589 23710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:56.634912 23710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431: Bootstrap starting.
I20260923 22:11:56.635507 23710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:56.635834 23710 log.cc:824] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:56.636574 23710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431: No bootstrap required, opened a new log
I20260923 22:11:56.638146 23710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f78fb2f9944435f95ba5a3f8452a431" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35065 } }
I20260923 22:11:56.638316 23710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:56.638365 23710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f78fb2f9944435f95ba5a3f8452a431, State: Initialized, Role: FOLLOWER
I20260923 22:11:56.638549 23710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f78fb2f9944435f95ba5a3f8452a431" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35065 } }
I20260923 22:11:56.638638 23710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:56.638684 23710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:56.638738 23710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:56.639647 23710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f78fb2f9944435f95ba5a3f8452a431" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35065 } }
I20260923 22:11:56.639772 23710 leader_election.cc:302] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f78fb2f9944435f95ba5a3f8452a431; no voters:
I20260923 22:11:56.639968 23710 leader_election.cc:288] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:56.640002 23713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:56.640182 23713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [term 1 LEADER]: Becoming Leader. State: Replica: 7f78fb2f9944435f95ba5a3f8452a431, State: Running, Role: LEADER
I20260923 22:11:56.640252 23710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:56.640321 23713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f78fb2f9944435f95ba5a3f8452a431" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35065 } }
I20260923 22:11:56.640784 23715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f78fb2f9944435f95ba5a3f8452a431. Latest consensus state: current_term: 1 leader_uuid: "7f78fb2f9944435f95ba5a3f8452a431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f78fb2f9944435f95ba5a3f8452a431" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35065 } } }
I20260923 22:11:56.640884 23715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:56.641135 23714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f78fb2f9944435f95ba5a3f8452a431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f78fb2f9944435f95ba5a3f8452a431" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35065 } } }
I20260923 22:11:56.641242 23714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:56.641335 23722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:56.641563 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 23642
I20260923 22:11:56.641687 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:56.641897 2003 external_mini_cluster.cc:1428] Setting key eaf8200862f1b272f1ab409ead72c301
I20260923 22:11:56.641896 23722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:56.643778 23722 catalog_manager.cc:1347] Generated new cluster ID: d7b1f741070b44818ffa554ae2246bba
I20260923 22:11:56.643831 23722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:56.648434 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:56.642502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38965 (local address 127.1.244.254:35065)
0923 22:11:56.642702 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:56.642712 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:56.642744 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:11:56.642865 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:56.642873 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:56.642932 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:56.643092 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:56.643457 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:56.644867 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:56.645961 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:56.646617 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:56.646742 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:56.647514 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:56.647518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:56.647518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:56.647782 (+ 264us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:56.647825 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:56.647841 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:56.647928 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:56.648159 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 22:11:56.648194 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260923 22:11:56.649451 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:56.654639 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:56.651009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49881 (local address 127.1.244.254:35065)
0923 22:11:56.651046 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:56.651048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:56.651187 (+ 139us) server_negotiation.cc:407] Connection header received
0923 22:11:56.651235 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:56.651236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:56.651246 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:56.651302 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:56.651435 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:56.652196 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:56.653195 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:56.653525 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:56.653586 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:56.654382 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:56.654383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:56.654384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:56.654388 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:56.654403 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:56.654408 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:56.654466 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:56.654549 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:11:56.654563 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:11:56.659881 23722 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:56.660595 23722 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:56.674966 23722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f78fb2f9944435f95ba5a3f8452a431: Generated new TSK 0
I20260923 22:11:56.675196 23722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:56.684417 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 23642
W20260923 22:11:56.688373 23627 connection.cc:535] client connection to 127.1.244.254:35065 recv error: Network error: failed to read from TLS socket (remote: 127.1.244.254:35065): Connection reset by peer (error 104)
2026-09-23T22:11:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:56 dist-test-slave-b5jz krb5kdc[23741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:56 dist-test-slave-b5jz krb5kdc[23741](info): set up 2 sockets
Sep 23 22:11:56 dist-test-slave-b5jz krb5kdc[23741](info): commencing operation
krb5kdc: starting...
W20260923 22:11:57.715760 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:57 dist-test-slave-b5jz krb5kdc[23741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:11:57.834339 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:44657
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:44657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:57.914088 23754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:57.914320 23754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:57.914359 23754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:57.915484 23754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:57.915558 23754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:57.915642 23754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:57.915674 23754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:57.915701 23754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:57.917524 23754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:44657
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:44657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:57.917788 23754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:57.918026 23754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:57.922713 23760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:57.922722 23754 server_base.cc:1034] running on GCE node
W20260923 22:11:57.922694 23759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:57.922691 23762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:57.923154 23754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:57.923457 23754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:57.924642 23754 hybrid_clock.cc:648] HybridClock initialized: now 1790201517924636 us; error 51 us; skew 500 ppm
Sep 23 22:11:57 dist-test-slave-b5jz krb5kdc[23741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201517, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:57.927083 23754 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:11:57.927624 23754 webserver.cc:460] Webserver started at http://127.1.244.254:34135/ using document root <none> and password file <none>
I20260923 22:11:57.927863 23754 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:57.927930 23754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:57.928084 23754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:57.929109 23754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "66c430aeb7f9480bbeeff743fa2db41e"
format_stamp: "Formatted at 2026-09-23 22:11:57 on dist-test-slave-b5jz"
server_key: "ce1fe0983fa7ea7f3e1b70bfb6ceb476"
server_key_iv: "660d5c2943be5daaf122d70c6d14106d"
server_key_version: "encryptionkey@0"
I20260923 22:11:57.929472 23754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "66c430aeb7f9480bbeeff743fa2db41e"
format_stamp: "Formatted at 2026-09-23 22:11:57 on dist-test-slave-b5jz"
server_key: "ce1fe0983fa7ea7f3e1b70bfb6ceb476"
server_key_iv: "660d5c2943be5daaf122d70c6d14106d"
server_key_version: "encryptionkey@0"
I20260923 22:11:57.930816 23754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:57.931624 23769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:57.931810 23754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:57.931910 23754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "66c430aeb7f9480bbeeff743fa2db41e"
format_stamp: "Formatted at 2026-09-23 22:11:57 on dist-test-slave-b5jz"
server_key: "ce1fe0983fa7ea7f3e1b70bfb6ceb476"
server_key_iv: "660d5c2943be5daaf122d70c6d14106d"
server_key_version: "encryptionkey@0"
I20260923 22:11:57.932003 23754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:57.935289 23754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:57.938344 23754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:57.938503 23754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:57.942550 23754 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:44657
I20260923 22:11:57.942606 23821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:44657 every 8 connection(s)
I20260923 22:11:57.942870 23754 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:57.943873 23822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:57.946131 23822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e: Bootstrap starting.
I20260923 22:11:57.946731 23822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:57.947018 23822 log.cc:824] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:57.947763 23822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e: No bootstrap required, opened a new log
I20260923 22:11:57.948477 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 23754
I20260923 22:11:57.948603 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:57.948798 2003 external_mini_cluster.cc:1428] Setting key e435cab2158dc05514315a959ce49e5c
I20260923 22:11:57.949875 23822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c430aeb7f9480bbeeff743fa2db41e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44657 } }
I20260923 22:11:57.950007 23822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:57.950030 23822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c430aeb7f9480bbeeff743fa2db41e, State: Initialized, Role: FOLLOWER
I20260923 22:11:57.950203 23822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c430aeb7f9480bbeeff743fa2db41e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44657 } }
I20260923 22:11:57.950315 23822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:57.950349 23822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:57.950397 23822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:57.951170 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:57.949365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35521 (local address 127.1.244.254:44657)
0923 22:11:57.949570 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:57.949581 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:57.949611 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:11:57.949857 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:57.949864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:57.949891 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:57.950124 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:57.950733 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:57.950737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:57.950737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:57.950741 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:57.950789 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:57.950801 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:57.950870 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:57.950947 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 22:11:57.950988 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
I20260923 22:11:57.951440 23822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c430aeb7f9480bbeeff743fa2db41e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44657 } }
I20260923 22:11:57.951583 23822 leader_election.cc:302] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66c430aeb7f9480bbeeff743fa2db41e; no voters:
I20260923 22:11:57.951754 23822 leader_election.cc:288] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:57.951782 23827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:57.952006 23827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [term 1 LEADER]: Becoming Leader. State: Replica: 66c430aeb7f9480bbeeff743fa2db41e, State: Running, Role: LEADER
I20260923 22:11:57.952013 23822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:57.952137 23827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c430aeb7f9480bbeeff743fa2db41e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44657 } }
I20260923 22:11:57.952194 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:57.952488 23827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66c430aeb7f9480bbeeff743fa2db41e. Latest consensus state: current_term: 1 leader_uuid: "66c430aeb7f9480bbeeff743fa2db41e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c430aeb7f9480bbeeff743fa2db41e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44657 } } }
I20260923 22:11:57.952548 23827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:57.953728 23828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66c430aeb7f9480bbeeff743fa2db41e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c430aeb7f9480bbeeff743fa2db41e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44657 } } }
W20260923 22:11:57.953786 23845 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260923 22:11:57.953863 23828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e [sys.catalog]: This master's current role is: LEADER
W20260923 22:11:57.953881 23845 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:11:57.954102 23835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:57.954298 23835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:57.956789 23835 catalog_manager.cc:1347] Generated new cluster ID: d8c20e35224a4a67805a1393a42f4a45
I20260923 22:11:57.956840 23835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:57.956902 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:57.954184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37609 (local address 127.1.244.254:44657)
0923 22:11:57.954233 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:57.954235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:57.955759 (+ 1524us) server_negotiation.cc:407] Connection header received
0923 22:11:57.955820 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:57.955821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:57.955826 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:57.955902 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:57.956647 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:57.956648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:57.956648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:57.956653 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:57.956666 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:57.956670 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:57.956700 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:57.956805 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:11:57.956817 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:11:57.963907 23835 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:57.964067 23835 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:57.966964 23835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66c430aeb7f9480bbeeff743fa2db41e: Generated new TSK 0
I20260923 22:11:57.967129 23835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:57.972020 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 23754
2026-09-23T22:11:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:58 dist-test-slave-b5jz krb5kdc[23853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:58 dist-test-slave-b5jz krb5kdc[23853](info): set up 2 sockets
Sep 23 22:11:58 dist-test-slave-b5jz krb5kdc[23853](info): commencing operation
krb5kdc: starting...
W20260923 22:11:59.007723 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:11:59 dist-test-slave-b5jz krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:11:59.124366 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46037
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:39025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:59.205003 23866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:59.205221 23866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:59.205245 23866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:59.206423 23866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:59.206467 23866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:59.206481 23866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:59.206494 23866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:59.206507 23866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:59.208096 23866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:39025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:11:59.208356 23866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:59.208618 23866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:59.213550 23871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:59.213557 23874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:59.213589 23866 server_base.cc:1034] running on GCE node
W20260923 22:11:59.213569 23872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:59.213964 23866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:59.214262 23866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:59.215430 23866 hybrid_clock.cc:648] HybridClock initialized: now 1790201519215406 us; error 43 us; skew 500 ppm
Sep 23 22:11:59 dist-test-slave-b5jz krb5kdc[23853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201519, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:59.217916 23866 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:11:59.218387 23866 webserver.cc:460] Webserver started at http://127.1.244.254:35339/ using document root <none> and password file <none>
I20260923 22:11:59.218624 23866 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:59.218698 23866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:59.218847 23866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:59.219954 23866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "90159a55efc84d8aba79636b6bfd20a4"
format_stamp: "Formatted at 2026-09-23 22:11:59 on dist-test-slave-b5jz"
server_key: "3ea08194422a11db2127aea2ede5a8cc"
server_key_iv: "c61b937a1e4e716e05b0534eb37c2d86"
server_key_version: "encryptionkey@0"
I20260923 22:11:59.220316 23866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "90159a55efc84d8aba79636b6bfd20a4"
format_stamp: "Formatted at 2026-09-23 22:11:59 on dist-test-slave-b5jz"
server_key: "3ea08194422a11db2127aea2ede5a8cc"
server_key_iv: "c61b937a1e4e716e05b0534eb37c2d86"
server_key_version: "encryptionkey@0"
I20260923 22:11:59.221607 23866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:59.222278 23881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:59.222461 23866 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:59.222563 23866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "90159a55efc84d8aba79636b6bfd20a4"
format_stamp: "Formatted at 2026-09-23 22:11:59 on dist-test-slave-b5jz"
server_key: "3ea08194422a11db2127aea2ede5a8cc"
server_key_iv: "c61b937a1e4e716e05b0534eb37c2d86"
server_key_version: "encryptionkey@0"
I20260923 22:11:59.222635 23866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:59.242146 23866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:59.244575 23866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:59.244750 23866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:59.248765 23866 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46037
I20260923 22:11:59.248801 23933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46037 every 8 connection(s)
I20260923 22:11:59.249202 23866 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:59.250084 23934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:59.252336 23934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4: Bootstrap starting.
I20260923 22:11:59.252868 23934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:59.253108 23934 log.cc:824] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:59.253755 23934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4: No bootstrap required, opened a new log
I20260923 22:11:59.255290 23934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90159a55efc84d8aba79636b6bfd20a4" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46037 } }
I20260923 22:11:59.255430 23934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:59.255453 23934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90159a55efc84d8aba79636b6bfd20a4, State: Initialized, Role: FOLLOWER
I20260923 22:11:59.255631 23934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90159a55efc84d8aba79636b6bfd20a4" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46037 } }
I20260923 22:11:59.255715 23934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:59.255739 23934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:59.255769 23934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:59.256549 23934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90159a55efc84d8aba79636b6bfd20a4" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46037 } }
I20260923 22:11:59.256634 23934 leader_election.cc:302] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90159a55efc84d8aba79636b6bfd20a4; no voters:
I20260923 22:11:59.256755 23934 leader_election.cc:288] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:59.256815 23937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:59.256989 23937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [term 1 LEADER]: Becoming Leader. State: Replica: 90159a55efc84d8aba79636b6bfd20a4, State: Running, Role: LEADER
I20260923 22:11:59.257053 23934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:59.257078 23937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90159a55efc84d8aba79636b6bfd20a4" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46037 } }
I20260923 22:11:59.257493 23938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90159a55efc84d8aba79636b6bfd20a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90159a55efc84d8aba79636b6bfd20a4" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46037 } } }
I20260923 22:11:59.257525 23939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90159a55efc84d8aba79636b6bfd20a4. Latest consensus state: current_term: 1 leader_uuid: "90159a55efc84d8aba79636b6bfd20a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90159a55efc84d8aba79636b6bfd20a4" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46037 } } }
I20260923 22:11:59.257649 23938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:59.257676 23939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:59.258039 23947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:59.258554 23947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:59.259253 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 23866
I20260923 22:11:59.259377 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:11:59.259569 2003 external_mini_cluster.cc:1428] Setting key 148aabbe68003bf10b0d8488c7cf82e6
I20260923 22:11:59.260265 23947 catalog_manager.cc:1347] Generated new cluster ID: 8916473db85d4850ba3eeb1f4e35784b
I20260923 22:11:59.260314 23947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:59 dist-test-slave-b5jz krb5kdc[23853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:11:59.271740 23957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:59.260070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33903 (local address 127.1.244.254:46037)
0923 22:11:59.260228 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:59.260238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:59.260509 (+ 271us) server_negotiation.cc:407] Connection header received
0923 22:11:59.260640 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:59.260647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:59.260702 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:59.260833 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:59.261645 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:59.263058 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:59.264232 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:59.264870 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:59.264972 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:59.267979 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:59.267984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:59.267984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:59.267996 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:59.270049 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:59.270429 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:59.270433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:59.270433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:59.270546 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:59.270752 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:59.270753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:59.270753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:59.271079 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:59.271252 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:59.271322 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:11:59.271459 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260923 22:11:59.273023 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:59.276085 23947 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:59.276751 23947 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:59.281584 23957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:59.274563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50569 (local address 127.1.244.254:46037)
0923 22:11:59.274597 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:59.274598 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:59.274606 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:11:59.274661 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:59.274661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:59.274671 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:59.274734 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:59.275316 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:59.276128 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:59.277064 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:59.277398 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:59.277471 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:59.278766 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:59.278767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:59.278767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:59.278774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:59.280362 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:59.280727 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:59.280728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:59.280728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:59.280790 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:59.280985 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:59.280986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:59.280986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:59.281113 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:59.281219 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:59.281311 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:11:59.281402 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:11:59.282608 23947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90159a55efc84d8aba79636b6bfd20a4: Generated new TSK 0
I20260923 22:11:59.282755 23947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:59.288800 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 23866
2026-09-23T22:11:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:59 dist-test-slave-b5jz krb5kdc[23965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:59 dist-test-slave-b5jz krb5kdc[23965](info): set up 2 sockets
Sep 23 22:11:59 dist-test-slave-b5jz krb5kdc[23965](info): commencing operation
krb5kdc: starting...
W20260923 22:12:00.344200 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:12:00 dist-test-slave-b5jz krb5kdc[23965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:12:00.461876 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:44961
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:44961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:00.542855 23978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:00.543111 23978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:00.543149 23978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:00.544368 23978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:00.544446 23978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:00.544471 23978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:00.544494 23978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:00.544519 23978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:00.546088 23978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:44961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:44961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:00.546382 23978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:00.546614 23978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:00.551728 23978 server_base.cc:1034] running on GCE node
W20260923 22:12:00.551651 23986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:00.551693 23984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:00.551673 23983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:00.552158 23978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:00.552486 23978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:00.553665 23978 hybrid_clock.cc:648] HybridClock initialized: now 1790201520553658 us; error 38 us; skew 500 ppm
Sep 23 22:12:00 dist-test-slave-b5jz krb5kdc[23965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:00.556488 23978 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:12:00.556948 23978 webserver.cc:460] Webserver started at http://127.1.244.254:39219/ using document root <none> and password file <none>
I20260923 22:12:00.557206 23978 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:00.557277 23978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:00.557435 23978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:00.558533 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "8ddf94b60d7a4ade948ce65abcbff390"
format_stamp: "Formatted at 2026-09-23 22:12:00 on dist-test-slave-b5jz"
server_key: "2e34d85754c55068f6324715c7af5319"
server_key_iv: "b75f6c6730ee2c9eddea565af87e0dcc"
server_key_version: "encryptionkey@0"
I20260923 22:12:00.558917 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "8ddf94b60d7a4ade948ce65abcbff390"
format_stamp: "Formatted at 2026-09-23 22:12:00 on dist-test-slave-b5jz"
server_key: "2e34d85754c55068f6324715c7af5319"
server_key_iv: "b75f6c6730ee2c9eddea565af87e0dcc"
server_key_version: "encryptionkey@0"
I20260923 22:12:00.560351 23978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:12:00.561124 23993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:00.561292 23978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:00.561401 23978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "8ddf94b60d7a4ade948ce65abcbff390"
format_stamp: "Formatted at 2026-09-23 22:12:00 on dist-test-slave-b5jz"
server_key: "2e34d85754c55068f6324715c7af5319"
server_key_iv: "b75f6c6730ee2c9eddea565af87e0dcc"
server_key_version: "encryptionkey@0"
I20260923 22:12:00.561476 23978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:00.576187 23978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:00.578470 23978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:00.578661 23978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:00.582708 23978 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:44961
I20260923 22:12:00.582769 24045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:44961 every 8 connection(s)
I20260923 22:12:00.583153 23978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:00.584117 24046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:00.586472 24046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390: Bootstrap starting.
I20260923 22:12:00.586820 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 23978
I20260923 22:12:00.586980 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:12:00.587160 24046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:00.587182 2003 external_mini_cluster.cc:1428] Setting key 041ef27d7eef7a42dc186d3fed857933
I20260923 22:12:00.587513 24046 log.cc:824] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:00.588362 24046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390: No bootstrap required, opened a new log
I20260923 22:12:00.590970 24046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddf94b60d7a4ade948ce65abcbff390" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44961 } }
I20260923 22:12:00.591142 24046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:00.591190 24046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ddf94b60d7a4ade948ce65abcbff390, State: Initialized, Role: FOLLOWER
I20260923 22:12:00.591323 24046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddf94b60d7a4ade948ce65abcbff390" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44961 } }
I20260923 22:12:00.591405 24046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:00.591449 24046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:00.591497 24046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:00.592283 24046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddf94b60d7a4ade948ce65abcbff390" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44961 } }
I20260923 22:12:00.592403 24046 leader_election.cc:302] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ddf94b60d7a4ade948ce65abcbff390; no voters:
I20260923 22:12:00.592576 24046 leader_election.cc:288] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:00.592689 24051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:00.592840 24046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:00.592878 24051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [term 1 LEADER]: Becoming Leader. State: Replica: 8ddf94b60d7a4ade948ce65abcbff390, State: Running, Role: LEADER
I20260923 22:12:00.593070 24051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddf94b60d7a4ade948ce65abcbff390" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44961 } }
I20260923 22:12:00.593556 24053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ddf94b60d7a4ade948ce65abcbff390. Latest consensus state: current_term: 1 leader_uuid: "8ddf94b60d7a4ade948ce65abcbff390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddf94b60d7a4ade948ce65abcbff390" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44961 } } }
I20260923 22:12:00.593640 24053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:00.594051 24052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ddf94b60d7a4ade948ce65abcbff390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddf94b60d7a4ade948ce65abcbff390" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 44961 } } }
I20260923 22:12:00.594337 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:00.587845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36979 (local address 127.1.244.254:44961)
0923 22:12:00.588012 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:00.588022 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:00.588057 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:12:00.588736 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:00.588746 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:00.588814 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:00.588980 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:00.589234 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:00.590736 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:00.591772 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:00.592434 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:00.592562 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:00.593370 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:00.593375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:00.593375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:00.593643 (+ 268us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:00.593693 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:00.593711 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:00.593774 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:00.593990 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 22:12:00.594069 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260923 22:12:00.594162 24052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390 [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:00.594939 24066 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:00.594980 24066 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:00.595057 24060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:00.595218 24060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:00.595589 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:00.596946 24060 catalog_manager.cc:1347] Generated new cluster ID: 965abe7ff2b04901ac0fa48a828f2fe9
I20260923 22:12:00.596999 24060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:00.602797 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:00.599705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59873 (local address 127.1.244.254:44961)
0923 22:12:00.599785 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:00.599788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:00.599797 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:00.599822 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:00.599822 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:00.599835 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:00.599905 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:00.600073 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:00.600627 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:00.601423 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:00.601656 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:00.601749 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:00.602525 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:00.602526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:00.602526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:00.602532 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:00.602548 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:00.602553 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:00.602599 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:00.602671 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:12:00.602693 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:12:00.613409 24060 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:00.613991 24060 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:00.618125 24060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ddf94b60d7a4ade948ce65abcbff390: Generated new TSK 0
I20260923 22:12:00.618315 24060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:00.627349 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 23978
2026-09-23T22:12:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:00 dist-test-slave-b5jz krb5kdc[24077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:00 dist-test-slave-b5jz krb5kdc[24077](info): set up 2 sockets
Sep 23 22:12:00 dist-test-slave-b5jz krb5kdc[24077](info): commencing operation
krb5kdc: starting...
W20260923 22:12:01.667757 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:12:01 dist-test-slave-b5jz krb5kdc[24077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:12:01.786825 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38653
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:01.867781 24090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:01.868016 24090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:01.868054 24090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:01.869303 24090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:01.869378 24090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:01.869402 24090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:01.869421 24090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:01.869441 24090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:01.870972 24090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:01.871266 24090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:01.871506 24090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:01.876520 24098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:01.876546 24096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:01.876520 24095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:01.876812 24090 server_base.cc:1034] running on GCE node
I20260923 22:12:01.877065 24090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:01.877372 24090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:01.878548 24090 hybrid_clock.cc:648] HybridClock initialized: now 1790201521878533 us; error 27 us; skew 500 ppm
Sep 23 22:12:01 dist-test-slave-b5jz krb5kdc[24077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:01.885545 24090 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:12:01.886035 24090 webserver.cc:460] Webserver started at http://127.1.244.254:46571/ using document root <none> and password file <none>
I20260923 22:12:01.886283 24090 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:01.886353 24090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:01.886503 24090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:01.887694 24090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "56118d2e35164b5e94113dfa21e1be7e"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-b5jz"
server_key: "4e72b84c084110bfd135823739928a7c"
server_key_iv: "95070216700363664253c797e77eb311"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.888053 24090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "56118d2e35164b5e94113dfa21e1be7e"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-b5jz"
server_key: "4e72b84c084110bfd135823739928a7c"
server_key_iv: "95070216700363664253c797e77eb311"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.889370 24090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:01.890081 24105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:01.890245 24090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:01.890353 24090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "56118d2e35164b5e94113dfa21e1be7e"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-b5jz"
server_key: "4e72b84c084110bfd135823739928a7c"
server_key_iv: "95070216700363664253c797e77eb311"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.890429 24090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:01.894008 24090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:01.896389 24090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:01.896550 24090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:01.900401 24090 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:38653
I20260923 22:12:01.900450 24157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:38653 every 8 connection(s)
I20260923 22:12:01.900792 24090 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:01.901378 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 24090
I20260923 22:12:01.901515 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:12:01.901736 2003 external_mini_cluster.cc:1428] Setting key 64589266226b3a95fb1fa81d13b8a056
I20260923 22:12:01.902021 24158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:01.904345 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.902224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43931 (local address 127.1.244.254:38653)
0923 22:12:01.902407 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.902415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.902440 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:12:01.903075 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.903082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.903120 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.903232 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.903908 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.903912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.903912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:01.903916 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.903950 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:01.903960 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.904015 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.904101 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.904162 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260923 22:12:01.905435 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:01.905555 24158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e: Bootstrap starting.
I20260923 22:12:01.906291 24158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:01.906616 24158 log.cc:824] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:01.907333 24158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e: No bootstrap required, opened a new log
I20260923 22:12:01.908703 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.907014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51935 (local address 127.1.244.254:38653)
0923 22:12:01.907046 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.907048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.907710 (+ 662us) server_negotiation.cc:407] Connection header received
0923 22:12:01.907770 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.907771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.907776 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.907822 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.908368 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.908369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.908369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:01.908373 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.908386 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:01.908390 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.908449 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.908529 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.908540 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:12:01.909776 24158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56118d2e35164b5e94113dfa21e1be7e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38653 } }
I20260923 22:12:01.909909 24158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:01.909931 24158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56118d2e35164b5e94113dfa21e1be7e, State: Initialized, Role: FOLLOWER
I20260923 22:12:01.910048 24158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56118d2e35164b5e94113dfa21e1be7e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38653 } }
I20260923 22:12:01.910133 24158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:01.910156 24158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:01.910184 24158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:01.910661 24158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56118d2e35164b5e94113dfa21e1be7e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38653 } }
I20260923 22:12:01.910750 24158 leader_election.cc:302] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56118d2e35164b5e94113dfa21e1be7e; no voters:
I20260923 22:12:01.910866 24158 leader_election.cc:288] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:01.910948 24168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:01.911091 24168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [term 1 LEADER]: Becoming Leader. State: Replica: 56118d2e35164b5e94113dfa21e1be7e, State: Running, Role: LEADER
I20260923 22:12:01.911135 24158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:01.911206 24168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56118d2e35164b5e94113dfa21e1be7e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38653 } }
I20260923 22:12:01.911597 24169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56118d2e35164b5e94113dfa21e1be7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56118d2e35164b5e94113dfa21e1be7e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38653 } } }
I20260923 22:12:01.911687 24169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:01.911850 24170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56118d2e35164b5e94113dfa21e1be7e. Latest consensus state: current_term: 1 leader_uuid: "56118d2e35164b5e94113dfa21e1be7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56118d2e35164b5e94113dfa21e1be7e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38653 } } }
I20260923 22:12:01.911931 24170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:01.911927 24174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:01.912441 24174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:01.914131 24174 catalog_manager.cc:1347] Generated new cluster ID: 5d117a0a04a843b5949409a183cfc95a
I20260923 22:12:01.914187 24174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:01.936444 24174 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:01.936668 24174 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:01.941768 24174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56118d2e35164b5e94113dfa21e1be7e: Generated new TSK 0
I20260923 22:12:01.941977 24174 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:01.945641 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 24090
2026-09-23T22:12:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1318 ms)
[----------] 18 tests from AuthTokenIssuingTest (23782 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:01 dist-test-slave-b5jz krb5kdc[24189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:02 dist-test-slave-b5jz krb5kdc[24189](info): set up 2 sockets
Sep 23 22:12:02 dist-test-slave-b5jz krb5kdc[24189](info): commencing operation
krb5kdc: starting...
W20260923 22:12:02.991860 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:12:03.109687 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35917
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35917,127.1.244.253:38897,127.1.244.252:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:03.189967 24202 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:03.190192 24202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:03.190210 24202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:03.190230 24202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:03.191304 24202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:03.191339 24202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:03.191354 24202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:03.191366 24202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:03.191378 24202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:03.192806 24202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35917,127.1.244.253:38897,127.1.244.252:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:03.192997 24202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:03.193162 24202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:03.198026 24208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:03.198036 24207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:03.198143 24202 server_base.cc:1034] running on GCE node
W20260923 22:12:03.198036 24210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:03.198455 24202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:03.198758 24202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:03.199926 24202 hybrid_clock.cc:648] HybridClock initialized: now 1790201523199917 us; error 40 us; skew 500 ppm
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:03.202345 24202 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:12:03.202781 24202 webserver.cc:460] Webserver started at http://127.1.244.254:39221/ using document root <none> and password file <none>
I20260923 22:12:03.203006 24202 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:03.203073 24202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:03.203222 24202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:03.204381 24202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "8a1a28132b4b4156b2645e029bcaa92d"
format_stamp: "Formatted at 2026-09-23 22:12:03 on dist-test-slave-b5jz"
server_key: "262e776bf6a9b5a5923221fd54c0f44c"
server_key_iv: "3c3f6e776bda1b19a2b1fc9e154aadd8"
server_key_version: "encryptionkey@0"
I20260923 22:12:03.204748 24202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "8a1a28132b4b4156b2645e029bcaa92d"
format_stamp: "Formatted at 2026-09-23 22:12:03 on dist-test-slave-b5jz"
server_key: "262e776bf6a9b5a5923221fd54c0f44c"
server_key_iv: "3c3f6e776bda1b19a2b1fc9e154aadd8"
server_key_version: "encryptionkey@0"
I20260923 22:12:03.206025 24202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:03.206719 24217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:03.206895 24202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:03.206996 24202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "8a1a28132b4b4156b2645e029bcaa92d"
format_stamp: "Formatted at 2026-09-23 22:12:03 on dist-test-slave-b5jz"
server_key: "262e776bf6a9b5a5923221fd54c0f44c"
server_key_iv: "3c3f6e776bda1b19a2b1fc9e154aadd8"
server_key_version: "encryptionkey@0"
I20260923 22:12:03.207068 24202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:03.237262 24202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:03.239435 24202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:03.239653 24202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:03.243584 24202 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35917
I20260923 22:12:03.243603 24269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35917 every 8 connection(s)
I20260923 22:12:03.244067 24202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:03.244222 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 24202
I20260923 22:12:03.244336 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:12:03.244539 2003 external_mini_cluster.cc:1428] Setting key 0c045d41dc839f8fb8180bd77eeade66
I20260923 22:12:03.245239 24270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:03.246851 24270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.244.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.253@KRBTEST.COM" created.
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:12:03.258879 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.247720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.1.244.254:35917)
0923 22:12:03.247900 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.247905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.247977 (+ 72us) server_negotiation.cc:407] Connection header received
0923 22:12:03.248566 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.248568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.248594 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.248705 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.249303 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.250146 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.251028 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.251336 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.251452 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.255445 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.255453 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.255453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.255468 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.257279 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.257662 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.257665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.257666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.257761 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.257980 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.257981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.257981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.258213 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.258309 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.258489 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.258611 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":109,"threads_started":1}
I20260923 22:12:03.258880 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.247637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35917 (local address 127.0.0.1:56344)
0923 22:12:03.247834 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.247884 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:03.248402 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:03.248716 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:03.248719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:03.249004 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:03.249202 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.249207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.250262 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.250263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.250933 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.250935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.251450 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.251451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.251518 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.252113 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:03.252121 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:03.255339 (+ 3218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:03.257379 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.257386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.257389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.257579 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.257831 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.257832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.257832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.257916 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.258287 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:03.258292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:03.258361 (+ 69us) client_negotiation.cc:769] Sending connection context
0923 22:12:03.258457 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 22:12:03.258690 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":87,"thread_start_us":94,"threads_started":1}
I20260923 22:12:03.259629 24270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:03.260006 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.259906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38897 (local address 127.0.0.1:59198)
0923 22:12:03.259941 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.259985 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260923 22:12:03.260147 24218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
I20260923 22:12:03.260669 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.260580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38897 (local address 127.0.0.1:59208)
0923 22:12:03.260616 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.260647 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260923 22:12:03.260833 24270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:38897: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
I20260923 22:12:03.270803 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:38897
--webserver_interface=127.1.244.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35917,127.1.244.253:38897,127.1.244.252:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:03.310041 24270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } attempt: 1
I20260923 22:12:03.310611 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.310431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38897 (local address 127.0.0.1:59224)
0923 22:12:03.310532 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.310583 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260923 22:12:03.311035 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.310949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38897 (local address 127.0.0.1:59230)
0923 22:12:03.310992 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.311017 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:12:03.311141 24270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:38897: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
W20260923 22:12:03.353825 24277 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:03.354048 24277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:03.354066 24277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:03.354087 24277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:03.355238 24277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:03.355284 24277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:03.355299 24277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:03.355312 24277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:03.355324 24277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:03.356880 24277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35917,127.1.244.253:38897,127.1.244.252:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:38897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.244.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:03.357136 24277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:03.357372 24277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:03.362213 24283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:03.362249 24282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:03.362213 24285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:03.362465 24277 server_base.cc:1034] running on GCE node
I20260923 22:12:03.362764 24277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:03.363013 24277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:03.364156 24277 hybrid_clock.cc:648] HybridClock initialized: now 1790201523364153 us; error 39 us; skew 500 ppm
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:03.366859 24277 init.cc:377] Logged in from keytab as kudu/127.1.244.253@KRBTEST.COM (short username kudu)
I20260923 22:12:03.367293 24277 webserver.cc:460] Webserver started at http://127.1.244.253:44235/ using document root <none> and password file <none>
I20260923 22:12:03.367488 24277 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:03.367530 24277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:03.367719 24277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:03.368870 24277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/data/instance:
uuid: "54c99a796eb442e880900ee20b3c26f4"
format_stamp: "Formatted at 2026-09-23 22:12:03 on dist-test-slave-b5jz"
server_key: "69db9056d7f1366347e4e94fb6405750"
server_key_iv: "3732774c2e25ad1b24fa6a671167af81"
server_key_version: "encryptionkey@0"
I20260923 22:12:03.369206 24277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/wal/instance:
uuid: "54c99a796eb442e880900ee20b3c26f4"
format_stamp: "Formatted at 2026-09-23 22:12:03 on dist-test-slave-b5jz"
server_key: "69db9056d7f1366347e4e94fb6405750"
server_key_iv: "3732774c2e25ad1b24fa6a671167af81"
server_key_version: "encryptionkey@0"
I20260923 22:12:03.370561 24277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:03.371309 24292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:03.371479 24277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:03.371606 24277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/wal
uuid: "54c99a796eb442e880900ee20b3c26f4"
format_stamp: "Formatted at 2026-09-23 22:12:03 on dist-test-slave-b5jz"
server_key: "69db9056d7f1366347e4e94fb6405750"
server_key_iv: "3732774c2e25ad1b24fa6a671167af81"
server_key_version: "encryptionkey@0"
I20260923 22:12:03.371677 24277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:03.379314 24270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } attempt: 2
I20260923 22:12:03.379832 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.379699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38897 (local address 127.0.0.1:59232)
0923 22:12:03.379764 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.379803 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:12:03.380173 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.380086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38897 (local address 127.0.0.1:59234)
0923 22:12:03.380125 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.380149 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 22:12:03.380270 24270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:38897: Network error: Client connection negotiation failed: client connection to 127.1.244.253:38897: connect: Connection refused (error 111)
I20260923 22:12:03.384187 24277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:03.386272 24277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:03.386432 24277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:03.390331 24277 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:38897
I20260923 22:12:03.390395 24344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:38897 every 8 connection(s)
I20260923 22:12:03.390758 24277 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
I20260923 22:12:03.391816 24345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:03.393352 24345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:03.396356 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 24277
I20260923 22:12:03.396440 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-1/wal/instance
I20260923 22:12:03.396646 2003 external_mini_cluster.cc:1428] Setting key 43f1ba7cfddb1c496dcec3659c6a7d7a
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:12:03.403067 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.394104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.1.244.254:35917)
0923 22:12:03.394184 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.394186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.394309 (+ 123us) server_negotiation.cc:407] Connection header received
0923 22:12:03.394822 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.394823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.394837 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.394902 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.395652 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.396144 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.397271 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.397616 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.397705 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.400139 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.400141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.400141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.400149 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.401625 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.402036 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.402037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.402038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.402100 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.402358 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.402359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.402359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.402483 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.402561 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.402865 (+ 304us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.402948 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 22:12:03.403255 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.394007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35917 (local address 127.0.0.1:56358)
0923 22:12:03.394182 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.394219 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:03.394712 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:03.394977 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:03.394980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:03.395271 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:03.395532 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.395536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.396254 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.396255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.397179 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.397182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.397742 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.397743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.397891 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.398432 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:03.398444 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:03.400039 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:03.401719 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.401726 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.401729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.401956 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.402173 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.402174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.402175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.402288 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.402552 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:03.402559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:03.402725 (+ 166us) client_negotiation.cc:769] Sending connection context
0923 22:12:03.402839 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 22:12:03.403004 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260923 22:12:03.403960 24345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.244.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.252@KRBTEST.COM" created.
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260923 22:12:03.411381 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.404243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38897 (local address 127.0.0.1:59248)
0923 22:12:03.404293 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.404304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:03.404345 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:03.404651 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:03.404652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:03.404762 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:03.404814 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.404815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.405684 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.405685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.406188 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.406189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.406660 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.406661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.406731 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.407130 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:03.407135 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:03.408334 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:03.410334 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.410335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.410335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.410494 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.410701 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.410702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.410702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.410760 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.411044 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:03.411045 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:03.411081 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 22:12:03.411160 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:12:03.411256 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:12:03.411445 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.404299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59248 (local address 127.1.244.253:38897)
0923 22:12:03.404483 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.404488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.404512 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:12:03.404536 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.404538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.404552 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.404643 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.404883 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.405586 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.406278 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.406584 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.406664 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.408433 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.408441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.408442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.408448 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.410242 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.410573 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.410576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.410576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.410634 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.410808 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.410809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.410809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.410989 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.411067 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.411189 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.411309 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260923 22:12:03.412081 24345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:03.412458 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.412313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37570)
0923 22:12:03.412368 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.412422 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 22:12:03.412619 24294 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
I20260923 22:12:03.413092 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.412976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37578)
0923 22:12:03.413035 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.413067 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 22:12:03.413211 24345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43241: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
I20260923 22:12:03.422494 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:43241
--webserver_interface=127.1.244.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35917,127.1.244.253:38897,127.1.244.252:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:03.462466 24345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } attempt: 1
I20260923 22:12:03.463042 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.462892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37588)
0923 22:12:03.462956 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.463005 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 22:12:03.463467 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.463336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37590)
0923 22:12:03.463410 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.463441 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260923 22:12:03.463622 24345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43241: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
I20260923 22:12:03.471439 24270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } attempt: 3
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260923 22:12:03.479606 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.471885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38897 (local address 127.0.0.1:59254)
0923 22:12:03.471955 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.471969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:03.472048 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:03.472223 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:03.472224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:03.472412 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:03.472479 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.472481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.473240 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.473242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.473863 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.473865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.474377 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.474378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.474470 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.475000 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:03.475005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:03.476655 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:03.478410 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.478411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.478412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.478605 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.478889 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.478890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.478890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.478959 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.479234 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:03.479235 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:03.479285 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 22:12:03.479375 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 22:12:03.479482 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 22:12:03.479606 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.471992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59254 (local address 127.1.244.253:38897)
0923 22:12:03.472069 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.472072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.472084 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:12:03.472115 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.472116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.472131 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.472218 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.472573 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.473130 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.473978 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.474289 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.474380 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.476759 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.476761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.476761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.476770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.478287 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.478752 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.478753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.478753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.478817 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.479025 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.479026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.479027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.479162 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.479243 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.479398 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.479482 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 22:12:03.480141 24270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:03.480525 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.480418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37594)
0923 22:12:03.480469 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.480497 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:12:03.480911 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.480817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37602)
0923 22:12:03.480864 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.480888 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:12:03.481031 24270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43241: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
W20260923 22:12:03.505839 24352 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:03.506055 24352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:03.506072 24352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:03.506093 24352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:03.507258 24352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:03.507303 24352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:03.507318 24352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:03.507332 24352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:03.507344 24352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:03.508879 24352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35917,127.1.244.253:38897,127.1.244.252:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:43241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.244.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:03.509131 24352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:03.509377 24352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:03.512166 24270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } attempt: 1
I20260923 22:12:03.512616 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.512457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37612)
0923 22:12:03.512524 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.512578 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:12:03.513374 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.512865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37622)
0923 22:12:03.513314 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.513351 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432}
W20260923 22:12:03.513525 24270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43241: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
W20260923 22:12:03.514271 24358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:03.514320 24357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:03.514285 24360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:03.514604 24352 server_base.cc:1034] running on GCE node
I20260923 22:12:03.514847 24352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:03.515120 24352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:03.516256 24352 hybrid_clock.cc:648] HybridClock initialized: now 1790201523516248 us; error 41 us; skew 500 ppm
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:03.518687 24352 init.cc:377] Logged in from keytab as kudu/127.1.244.252@KRBTEST.COM (short username kudu)
I20260923 22:12:03.519146 24352 webserver.cc:460] Webserver started at http://127.1.244.252:39861/ using document root <none> and password file <none>
I20260923 22:12:03.519327 24352 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:03.519367 24352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:03.519454 24352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:03.520505 24352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/data/instance:
uuid: "74860cd14f0243019ececa9f4e9638c4"
format_stamp: "Formatted at 2026-09-23 22:12:03 on dist-test-slave-b5jz"
server_key: "61a5b8f588d6e9d1306e097d276ee0ce"
server_key_iv: "b03aba69e2da9948a350f14bff8acd16"
server_key_version: "encryptionkey@0"
I20260923 22:12:03.520810 24352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/wal/instance:
uuid: "74860cd14f0243019ececa9f4e9638c4"
format_stamp: "Formatted at 2026-09-23 22:12:03 on dist-test-slave-b5jz"
server_key: "61a5b8f588d6e9d1306e097d276ee0ce"
server_key_iv: "b03aba69e2da9948a350f14bff8acd16"
server_key_version: "encryptionkey@0"
I20260923 22:12:03.522017 24352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:03.522751 24367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:03.522940 24352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:03.523046 24352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/wal
uuid: "74860cd14f0243019ececa9f4e9638c4"
format_stamp: "Formatted at 2026-09-23 22:12:03 on dist-test-slave-b5jz"
server_key: "61a5b8f588d6e9d1306e097d276ee0ce"
server_key_iv: "b03aba69e2da9948a350f14bff8acd16"
server_key_version: "encryptionkey@0"
I20260923 22:12:03.523121 24352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:03.531778 24345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } attempt: 2
I20260923 22:12:03.532225 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.532095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37628)
0923 22:12:03.532155 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.532197 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 22:12:03.532663 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.532525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37630)
0923 22:12:03.532602 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.532640 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 22:12:03.532817 24345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43241: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43241: connect: Connection refused (error 111)
I20260923 22:12:03.539860 24352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:03.542166 24352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:03.542337 24352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:03.546437 24352 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:43241
I20260923 22:12:03.546471 24419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:43241 every 8 connection(s)
I20260923 22:12:03.546866 24352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
I20260923 22:12:03.547427 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 24352
I20260923 22:12:03.547546 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201256447402-2003-0/minicluster-data/master-2/wal/instance
I20260923 22:12:03.547768 2003 external_mini_cluster.cc:1428] Setting key 4b8f92dfa2fcc3fb1a4423570d44cae4
I20260923 22:12:03.547964 24420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:03.550567 24420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:12:03.558670 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.548299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37638 (local address 127.1.244.252:43241)
0923 22:12:03.548474 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.548490 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.548516 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:12:03.549183 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.549196 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.549257 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.549383 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.550236 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.551406 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.552324 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.552752 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.552872 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.555250 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.555257 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.555257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.555263 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.557048 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.557481 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.557484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.557484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.557567 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.557795 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.557795 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.557796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.558009 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.558183 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.558298 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.558423 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260923 22:12:03.558763 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.551240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56362 (local address 127.1.244.254:35917)
0923 22:12:03.551311 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.551314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.551465 (+ 151us) server_negotiation.cc:407] Connection header received
0923 22:12:03.551536 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.551537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.551572 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.551642 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.552089 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.552545 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.553474 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.553678 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.553785 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.555946 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.555947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.555947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.555955 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.557500 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.557938 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.557939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.557939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.557994 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.558208 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.558209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.558209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.558325 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.558399 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.558586 (+ 187us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.558666 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:12:03.558805 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.551175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35917 (local address 127.0.0.1:56362)
0923 22:12:03.551362 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.551400 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:03.551482 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:03.551660 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:03.551662 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:03.551931 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:03.551996 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.551997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.552638 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.552640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.553386 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.553389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.553814 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.553816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.553885 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.554346 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:03.554354 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:03.555857 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:03.557585 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.557591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.557593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.557866 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.558071 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.558072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.558072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.558140 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.558392 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:03.558398 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:03.558451 (+ 53us) client_negotiation.cc:769] Sending connection context
0923 22:12:03.558558 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 22:12:03.558666 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260923 22:12:03.559432 24420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:12:03.567476 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.559780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59256 (local address 127.1.244.253:38897)
0923 22:12:03.559826 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.559828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.559838 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:03.559863 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.559864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.559874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.559930 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.560212 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.560831 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.561569 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.561904 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.561964 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.564495 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.564497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.564497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.564506 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.566282 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.566661 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.566661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.566662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.566713 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.566934 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.566934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.566935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.567080 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.567158 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.567316 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.567396 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:03.567472 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.559723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38897 (local address 127.0.0.1:59256)
0923 22:12:03.559765 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.559775 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:03.559813 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:03.559930 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:03.559931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:03.560083 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:03.560121 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.560129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.560905 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.560906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.561494 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.561496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.561972 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.561973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.562031 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.562413 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:03.562419 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:03.563797 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:03.566364 (+ 2567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.566365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.566366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.566584 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.566815 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.566815 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.566816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.566868 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.567148 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:03.567149 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:03.567200 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 22:12:03.567270 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:12:03.567371 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:12:03.567902 24420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:03.568500 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.559764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56374 (local address 127.1.244.254:35917)
0923 22:12:03.559798 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.559800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.559870 (+ 70us) server_negotiation.cc:407] Connection header received
0923 22:12:03.559940 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.559941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.559950 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.560010 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.560558 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.561107 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.561872 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.562137 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.562305 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.565091 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.565092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.565093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.565099 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.566695 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.567070 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.567071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.567071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.567141 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.567377 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.567378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.567378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.567913 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.568294 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.568343 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.568429 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260923 22:12:03.574589 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.568165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37642)
0923 22:12:03.568216 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.568227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:03.568263 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:03.568412 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:03.568413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:03.568561 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:03.568604 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.568606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.569389 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.569390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.569866 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.569867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.570235 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.570235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.570278 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.570693 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:03.570698 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:03.571933 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:03.573442 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.573443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.573443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.573672 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.573873 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.573874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.573874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.573934 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.574211 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:03.574212 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:03.574287 (+ 75us) client_negotiation.cc:769] Sending connection context
0923 22:12:03.574352 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:12:03.574456 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 22:12:03.574607 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.568225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37642 (local address 127.1.244.252:43241)
0923 22:12:03.568325 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.568327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.568332 (+ 5us) server_negotiation.cc:407] Connection header received
0923 22:12:03.568348 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.568348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.568355 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.568402 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.568697 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.569313 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.569955 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.570167 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.570240 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.572015 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.572016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.572017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.572024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.573359 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.573746 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.573747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.573747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.573795 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.573995 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.573996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.573996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.574126 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.574189 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.574400 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.574504 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":85}
I20260923 22:12:03.576465 24420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4: Bootstrap starting.
I20260923 22:12:03.576987 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.569178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.1.244.253:38897)
0923 22:12:03.569228 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.569230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.569239 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:03.569261 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.569262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.569271 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.569325 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.569789 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.570237 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.570896 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.571171 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.571243 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.573828 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.573830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.573830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.573837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.575819 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.576195 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.576196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.576197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.576253 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.576468 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.576469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.576469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.576612 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.576778 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.576814 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.576888 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:03.577477 24420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:03.577821 24420 log.cc:824] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:03.578622 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:03.578840 24420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4: No bootstrap required, opened a new log
I20260923 22:12:03.579934 24244 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8a1a28132b4b4156b2645e029bcaa92d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56374
I20260923 22:12:03.580523 24420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } }
I20260923 22:12:03.580636 24420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74860cd14f0243019ececa9f4e9638c4, State: Initialized, Role: FOLLOWER
I20260923 22:12:03.580726 24420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } }
I20260923 22:12:03.581063 24420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:03.581045 24426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.581182 24426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:12:03.582881 24437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:03.582980 24437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:03.583246 24352 master_runner.cc:186] Error getting master registration for 127.1.244.254:35917: OK, instance_id { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" instance_seqno: 1790201523242477 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:12:03.585359 24352 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:12:03.588775 24270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } attempt: 2
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260923 22:12:03.596563 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.589053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37646)
0923 22:12:03.589119 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.589130 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:03.589179 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:03.589413 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:03.589415 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:03.589584 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:03.589628 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.589629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.590277 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.590278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.590790 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.590792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.591169 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.591170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.591232 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.591771 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:03.591777 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:03.593371 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:03.595224 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.595225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.595226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.595467 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.595693 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.595822 (+ 129us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.595823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.595899 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.596223 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:03.596224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:03.596276 (+ 52us) client_negotiation.cc:769] Sending connection context
0923 22:12:03.596352 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 22:12:03.596455 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260923 22:12:03.596563 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.589240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.1.244.252:43241)
0923 22:12:03.589298 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.589300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.589315 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:12:03.589334 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.589335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.589342 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.589407 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.589716 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.590188 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.590876 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.591067 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.591157 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.593472 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.593473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.593474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.593482 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.595134 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.595565 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.595566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.595566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.595623 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.595969 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.595970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.595970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.596149 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.596217 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.596388 (+ 171us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.596466 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:12:03.598246 24270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d: Bootstrap starting.
I20260923 22:12:03.599009 24270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:03.599308 24270 log.cc:824] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:03.600217 24270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d: No bootstrap required, opened a new log
I20260923 22:12:03.601573 24270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } }
I20260923 22:12:03.601734 24270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a28132b4b4156b2645e029bcaa92d, State: Initialized, Role: FOLLOWER
I20260923 22:12:03.601822 24270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } }
I20260923 22:12:03.602145 24270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:03.602174 24244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 22:12:03.602108 24440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.602285 24440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:03.602321 24244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:03.603147 24244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } }
I20260923 22:12:03.603864 24394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a1a28132b4b4156b2645e029bcaa92d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "74860cd14f0243019ececa9f4e9638c4"
I20260923 22:12:03.603740 24244 leader_election.cc:288] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [CANDIDATE]: Term 1 election: Requested vote from peers 54c99a796eb442e880900ee20b3c26f4 (127.1.244.253:38897), 74860cd14f0243019ececa9f4e9638c4 (127.1.244.252:43241)
I20260923 22:12:03.604008 24394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [term 0 FOLLOWER]: Advancing to term 1
W20260923 22:12:03.604595 24451 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:03.604697 24451 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:03.604976 24394 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a28132b4b4156b2645e029bcaa92d in term 1.
I20260923 22:12:03.605571 24219 leader_election.cc:302] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74860cd14f0243019ececa9f4e9638c4, 8a1a28132b4b4156b2645e029bcaa92d; no voters:
I20260923 22:12:03.605748 24440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:03.605890 24440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [term 1 LEADER]: Becoming Leader. State: Replica: 8a1a28132b4b4156b2645e029bcaa92d, State: Running, Role: LEADER
I20260923 22:12:03.605980 24440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } }
I20260923 22:12:03.606273 24202 master_runner.cc:186] Error getting master registration for 127.1.244.253:38897: OK, instance_id { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" instance_seqno: 1790201523389300 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:12:03.608776 24202 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:12:03.615056 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.606456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.1.244.253:38897)
0923 22:12:03.606506 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.606508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.606802 (+ 294us) server_negotiation.cc:407] Connection header received
0923 22:12:03.606900 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.606901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.606910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.606940 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.607770 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.608519 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.609568 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.609922 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.609966 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.611706 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.611707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.611708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.611715 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.613814 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.614227 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.614228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.614229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.614288 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.614476 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.614477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.614477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.614605 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.614675 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.614876 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.614957 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:03.616506 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.605716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56376 (local address 127.1.244.254:35917)
0923 22:12:03.606409 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.606411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.606421 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:03.606442 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.606443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.606452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.606511 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.609690 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.610369 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.611387 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.611692 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.611739 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.613172 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.613173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.613173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.613181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.614901 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.615615 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.615617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.615617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.615679 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.615913 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.615914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.615914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.616052 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.616122 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.616332 (+ 210us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.616416 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":673}
I20260923 22:12:03.617461 24458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a1a28132b4b4156b2645e029bcaa92d. Latest consensus state: current_term: 1 leader_uuid: "8a1a28132b4b4156b2645e029bcaa92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.617664 24458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:03.618009 24462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:03.618247 24462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:03.620254 24394 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [term 1 FOLLOWER]: Refusing update from remote peer 8a1a28132b4b4156b2645e029bcaa92d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:03.620658 24458 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [LEADER]: Connected to new peer: Peer: permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:03.622463 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.606903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.1.244.252:43241)
0923 22:12:03.608856 (+ 1953us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.608858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.615376 (+ 6518us) server_negotiation.cc:407] Connection header received
0923 22:12:03.615475 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.615477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.615487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.615517 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.616120 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.616849 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.617848 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.618154 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.618198 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.619652 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.619654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.619654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.619662 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.621303 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.621659 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.621660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.621660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.621714 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.621898 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.621898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.621899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.622020 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.622094 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.622307 (+ 213us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.622378 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1927}
I20260923 22:12:03.622843 24426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a1a28132b4b4156b2645e029bcaa92d. Latest consensus state: current_term: 1 leader_uuid: "8a1a28132b4b4156b2645e029bcaa92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.622972 24426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:03.623934 24345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } attempt: 3
Sep 23 22:12:03 dist-test-slave-b5jz krb5kdc[24189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260923 22:12:03.632346 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.624174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43241 (local address 127.0.0.1:37662)
0923 22:12:03.624213 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:03.624225 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:03.624267 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:03.624409 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:03.624410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:03.624579 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:03.624626 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.624628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.625501 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.625502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.626366 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:03.626369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.626939 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.626940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:03.627020 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.627446 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:03.627451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:03.629015 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:03.630994 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.630996 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.630996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.631218 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.631519 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:03.631520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:03.631521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:03.631623 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:03.632021 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:03.632022 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:03.632071 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:12:03.632140 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:12:03.632242 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 22:12:03.632622 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:03.624240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.1.244.252:43241)
0923 22:12:03.624294 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.624296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.624304 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:03.624323 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.624324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.624333 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.624382 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.624694 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.625415 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.626465 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:03.626774 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:03.626837 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:03.629109 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:03.629111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:03.629111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:03.629119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:03.630900 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.631298 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.631298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.631299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.631420 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:03.631707 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:03.631708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:03.631708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:03.631923 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:03.632028 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:03.632466 (+ 438us) server_negotiation.cc:299] Negotiation successful
0923 22:12:03.632536 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:12:03.634307 24345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4: Bootstrap starting.
I20260923 22:12:03.635025 24345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:03.635342 24345 log.cc:824] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:03.636220 24345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4: No bootstrap required, opened a new log
I20260923 22:12:03.638206 24345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } }
I20260923 22:12:03.638343 24345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c99a796eb442e880900ee20b3c26f4, State: Initialized, Role: FOLLOWER
I20260923 22:12:03.638417 24345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } }
I20260923 22:12:03.638716 24345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:03.638818 24318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:03.639633 24319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a1a28132b4b4156b2645e029bcaa92d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "54c99a796eb442e880900ee20b3c26f4"
W20260923 22:12:03.639870 24298 tablet.cc:2369] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:03.640030 24318 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [term 1 FOLLOWER]: Refusing update from remote peer 8a1a28132b4b4156b2645e029bcaa92d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:03.640483 24458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a1a28132b4b4156b2645e029bcaa92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.640580 24458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:03.640707 24440 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:03.640955 24319 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a28132b4b4156b2645e029bcaa92d in term 1.
I20260923 22:12:03.641376 24426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a1a28132b4b4156b2645e029bcaa92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.641480 24426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:03.643361 24476 mvcc.cc:205] Tried to move back new op lower bound from 7332665440746008576 to 7332665440690757632. Current Snapshot: MvccSnapshot[applied={T|T < 7332665440746008576}]
I20260923 22:12:03.643728 24463 mvcc.cc:205] Tried to move back new op lower bound from 7332665440746008576 to 7332665440690757632. Current Snapshot: MvccSnapshot[applied={T|T < 7332665440746008576}]
I20260923 22:12:03.644018 24467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a1a28132b4b4156b2645e029bcaa92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.644116 24467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:03.644205 24467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a1a28132b4b4156b2645e029bcaa92d. Latest consensus state: current_term: 1 leader_uuid: "8a1a28132b4b4156b2645e029bcaa92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.644263 24467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:03.644292 24467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a1a28132b4b4156b2645e029bcaa92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.644341 24467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:03.644868 24462 catalog_manager.cc:1347] Generated new cluster ID: 5c0e8af5d42b494cb34f363c8623292c
I20260923 22:12:03.644927 24462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260923 22:12:03.645069 24485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:03.645135 24485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:03.684186 24440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a1a28132b4b4156b2645e029bcaa92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a28132b4b4156b2645e029bcaa92d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35917 } } peers { permanent_uuid: "54c99a796eb442e880900ee20b3c26f4" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38897 } } peers { permanent_uuid: "74860cd14f0243019ececa9f4e9638c4" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43241 } } }
I20260923 22:12:03.684332 24440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:03.686101 24462 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:03.686861 24462 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:03.694119 24462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a1a28132b4b4156b2645e029bcaa92d: Generated new TSK 0
I20260923 22:12:03.694317 24462 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:12:03.722543 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.721469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.1.244.253:38897)
0923 22:12:03.721518 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.721521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.721839 (+ 318us) server_negotiation.cc:407] Connection header received
0923 22:12:03.721948 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.721949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.721960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.721992 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.722464 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59286: BlockingRecv error: recv got EOF from 127.0.0.1:59286 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260923 22:12:03.724862 24423 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.723852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.1.244.252:43241)
0923 22:12:03.723895 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.723896 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.724361 (+ 465us) server_negotiation.cc:407] Connection header received
0923 22:12:03.724388 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.724389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.724399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.724442 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.724797 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37676: BlockingRecv error: recv got EOF from 127.0.0.1:37676 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 22:12:03.725698 24274 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.724098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56392 (local address 127.1.244.254:35917)
0923 22:12:03.724138 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.724140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.725184 (+ 1044us) server_negotiation.cc:407] Connection header received
0923 22:12:03.725212 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.725214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.725225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.725271 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.725621 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56392: BlockingRecv error: recv got EOF from 127.0.0.1:56392 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:12:03.728212 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.727355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59294 (local address 127.1.244.253:38897)
0923 22:12:03.727391 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.727393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.727637 (+ 244us) server_negotiation.cc:407] Connection header received
0923 22:12:03.727769 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.727770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.727790 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.727822 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.728164 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59294: BlockingRecv error: recv got EOF from 127.0.0.1:59294 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:12:03.731263 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.730515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59298 (local address 127.1.244.253:38897)
0923 22:12:03.730549 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.730551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.730779 (+ 228us) server_negotiation.cc:407] Connection header received
0923 22:12:03.730873 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.730874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.730884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.730912 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.731215 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59298: BlockingRecv error: recv got EOF from 127.0.0.1:59298 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:12:03.735301 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.734571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.1.244.253:38897)
0923 22:12:03.734606 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.734608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.734823 (+ 215us) server_negotiation.cc:407] Connection header received
0923 22:12:03.734918 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.734919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.734929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.734957 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.735258 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59312: BlockingRecv error: recv got EOF from 127.0.0.1:59312 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:12:03.741294 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.740505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.1.244.253:38897)
0923 22:12:03.740540 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.740542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.740777 (+ 235us) server_negotiation.cc:407] Connection header received
0923 22:12:03.740868 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.740868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.740879 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.740906 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.741246 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59324: BlockingRecv error: recv got EOF from 127.0.0.1:59324 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:12:03.751540 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.750622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59326 (local address 127.1.244.253:38897)
0923 22:12:03.750666 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.750668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.750946 (+ 278us) server_negotiation.cc:407] Connection header received
0923 22:12:03.751053 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.751055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.751067 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.751098 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.751481 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59326: BlockingRecv error: recv got EOF from 127.0.0.1:59326 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:12:03.770001 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.769039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.1.244.253:38897)
0923 22:12:03.769087 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.769089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.769339 (+ 250us) server_negotiation.cc:407] Connection header received
0923 22:12:03.769442 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.769443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.769454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.769485 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.769936 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59330: BlockingRecv error: recv got EOF from 127.0.0.1:59330 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 22:12:03.804323 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.803454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.1.244.253:38897)
0923 22:12:03.803503 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.803505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.803692 (+ 187us) server_negotiation.cc:407] Connection header received
0923 22:12:03.803804 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.803805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.803817 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.803885 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.804247 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59340: BlockingRecv error: recv got EOF from 127.0.0.1:59340 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 22:12:03.870182 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.869474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.1.244.253:38897)
0923 22:12:03.869550 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.869552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.869649 (+ 97us) server_negotiation.cc:407] Connection header received
0923 22:12:03.869706 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.869707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.869718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.869792 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:03.870106 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59354: BlockingRecv error: recv got EOF from 127.0.0.1:59354 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260923 22:12:04.000293 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:03.999401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59368 (local address 127.1.244.253:38897)
0923 22:12:03.999463 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:03.999465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:03.999598 (+ 133us) server_negotiation.cc:407] Connection header received
0923 22:12:03.999697 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:03.999698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:03.999709 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:03.999773 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.000188 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59368: BlockingRecv error: recv got EOF from 127.0.0.1:59368 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260923 22:12:04.258251 24349 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.257572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.1.244.253:38897)
0923 22:12:04.257642 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.257643 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.257670 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:12:04.257743 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.257744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.257754 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.257820 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.258139 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59382: BlockingRecv error: recv got EOF from 127.0.0.1:59382 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:12:04.583461 24437 catalog_manager.cc:1259] Loaded cluster ID: 5c0e8af5d42b494cb34f363c8623292c
I20260923 22:12:04.583536 24437 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4: loading cluster ID for follower catalog manager: success
I20260923 22:12:04.584416 24437 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4: acquiring CA information for follower catalog manager: success
I20260923 22:12:04.584679 24437 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 74860cd14f0243019ececa9f4e9638c4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:12:04.645560 24485 catalog_manager.cc:1259] Loaded cluster ID: 5c0e8af5d42b494cb34f363c8623292c
I20260923 22:12:04.645642 24485 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4: loading cluster ID for follower catalog manager: success
I20260923 22:12:04.646502 24485 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4: acquiring CA information for follower catalog manager: success
I20260923 22:12:04.646831 24485 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 54c99a796eb442e880900ee20b3c26f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:12:04.774264 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.771451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.1.244.253:38897)
0923 22:12:04.771591 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.771593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.771607 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:12:04.771667 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.771668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.771741 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.771828 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.771971 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.772518 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.773304 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.773518 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.773595 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.773882 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.774004 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.774070 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.774118 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.774137 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260923 22:12:04.778913 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.776423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37680 (local address 127.1.244.252:43241)
0923 22:12:04.776533 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.776534 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.776542 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:04.776597 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.776598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.776632 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.776703 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.776834 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.777309 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.778208 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.778407 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.778482 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.778619 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.778731 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.778794 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.778831 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.778850 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":63,"threads_started":1}
I20260923 22:12:04.784102 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.781000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.1.244.253:38897)
0923 22:12:04.781046 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.781048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.781182 (+ 134us) server_negotiation.cc:407] Connection header received
0923 22:12:04.781220 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.781221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.781233 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.781287 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.781398 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.781986 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.783197 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.783483 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.783569 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.783742 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.783778 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.783829 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.783946 (+ 117us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.783972 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:04.784214 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.780891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37682 (local address 127.1.244.252:43241)
0923 22:12:04.780939 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.780943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.781171 (+ 228us) server_negotiation.cc:407] Connection header received
0923 22:12:04.781269 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.781270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.781282 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.781328 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.781467 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.782216 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.783320 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.783634 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.783703 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.783950 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.783980 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.784030 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.784100 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.784125 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:04.784591 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.780869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.1.244.254:35917)
0923 22:12:04.781019 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.781022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.781320 (+ 298us) server_negotiation.cc:407] Connection header received
0923 22:12:04.781369 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.781371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.781422 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.781519 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.781726 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.782477 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.783280 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.783622 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.783672 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.783946 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.784060 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.784117 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.784481 (+ 364us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.784506 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260923 22:12:04.786151 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 24202
I20260923 22:12:04.791992 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 24277
I20260923 22:12:04.796936 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 24352
2026-09-23T22:12:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2853 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:04 dist-test-slave-b5jz krb5kdc[24566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:04 dist-test-slave-b5jz krb5kdc[24566](info): set up 2 sockets
Sep 23 22:12:04 dist-test-slave-b5jz krb5kdc[24566](info): commencing operation
krb5kdc: starting...
W20260923 22:12:05.839900 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/test-user.keytab.
Sep 23 22:12:05 dist-test-slave-b5jz krb5kdc[24566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260923 22:12:05.958527 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45739
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:45739,127.1.244.253:35329,127.1.244.252:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:06.042261 24579 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:06.042492 24579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:06.042510 24579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:06.042532 24579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:06.043664 24579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:06.043704 24579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:06.043717 24579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:06.043730 24579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:06.043742 24579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:06.045182 24579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:45739,127.1.244.253:35329,127.1.244.252:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:06.045403 24579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:06.045590 24579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:06.050494 24587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:06.050485 24585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:06.050485 24584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:06.050568 24579 server_base.cc:1034] running on GCE node
I20260923 22:12:06.050942 24579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:06.051278 24579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:06.052455 24579 hybrid_clock.cc:648] HybridClock initialized: now 1790201526052446 us; error 40 us; skew 500 ppm
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:06.054906 24579 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260923 22:12:06.055359 24579 webserver.cc:460] Webserver started at http://127.1.244.254:46505/ using document root <none> and password file <none>
I20260923 22:12:06.055670 24579 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:06.055739 24579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:06.055879 24579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:06.056947 24579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/data/instance:
uuid: "d35eaf617af048de975333061be8f725"
format_stamp: "Formatted at 2026-09-23 22:12:06 on dist-test-slave-b5jz"
server_key: "e520c2e6960d8df35dc05d97ad4a0592"
server_key_iv: "b56edb01a0477a8ab18f3d88ac8c51b1"
server_key_version: "encryptionkey@0"
I20260923 22:12:06.057330 24579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/wal/instance:
uuid: "d35eaf617af048de975333061be8f725"
format_stamp: "Formatted at 2026-09-23 22:12:06 on dist-test-slave-b5jz"
server_key: "e520c2e6960d8df35dc05d97ad4a0592"
server_key_iv: "b56edb01a0477a8ab18f3d88ac8c51b1"
server_key_version: "encryptionkey@0"
I20260923 22:12:06.058725 24579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:06.059489 24594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:06.059711 24579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:06.059818 24579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/wal
uuid: "d35eaf617af048de975333061be8f725"
format_stamp: "Formatted at 2026-09-23 22:12:06 on dist-test-slave-b5jz"
server_key: "e520c2e6960d8df35dc05d97ad4a0592"
server_key_iv: "b56edb01a0477a8ab18f3d88ac8c51b1"
server_key_version: "encryptionkey@0"
I20260923 22:12:06.059893 24579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:06.079104 24579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:06.081166 24579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:06.081337 24579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:06.085458 24579 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:45739
I20260923 22:12:06.085498 24646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:45739 every 8 connection(s)
I20260923 22:12:06.085870 24579 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:06.086890 24647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:06.088438 24647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:06.093582 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 24579
I20260923 22:12:06.093703 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-0/wal/instance
I20260923 22:12:06.093926 2003 external_mini_cluster.cc:1428] Setting key cf0ae8ccbc27a7d977ea77bd87602fb8
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:12:06.098027 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.089097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45739 (local address 127.0.0.1:60034)
0923 22:12:06.089259 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.089301 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:06.089751 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:06.090011 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:06.090014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:06.090258 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:06.090416 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.090420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.091390 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.091391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.092112 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.092114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.092578 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.092579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.092629 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.093170 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:06.093178 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:06.094719 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:06.096533 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.096538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.096540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.096731 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.096977 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.096978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.096979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.097068 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.097433 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:06.097437 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:06.097499 (+ 62us) client_negotiation.cc:769] Sending connection context
0923 22:12:06.097604 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:12:06.097836 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":93,"thread_start_us":89,"threads_started":1}
I20260923 22:12:06.098030 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.089152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.1.244.254:45739)
0923 22:12:06.089310 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.089314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.089402 (+ 88us) server_negotiation.cc:407] Connection header received
0923 22:12:06.089875 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.089877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.089900 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.089985 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.090495 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.091307 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.092178 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.092488 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.092565 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.094814 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.094821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.094821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.094828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.096410 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.096821 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.096824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.096825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.096907 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.097141 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.097142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.097142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.097353 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.097439 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.097635 (+ 196us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.097771 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260923 22:12:06.098819 24647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:06.099181 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.099061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35329 (local address 127.0.0.1:33172)
0923 22:12:06.099109 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.099152 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:12:06.099299 24595 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
I20260923 22:12:06.099749 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.099661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35329 (local address 127.0.0.1:33182)
0923 22:12:06.099704 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.099728 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:12:06.099885 24647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:35329: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.244.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
I20260923 22:12:06.119257 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:35329
--webserver_interface=127.1.244.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:45739,127.1.244.253:35329,127.1.244.252:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:06.149104 24647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } attempt: 1
I20260923 22:12:06.149595 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.149455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35329 (local address 127.0.0.1:33198)
0923 22:12:06.149525 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.149565 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 22:12:06.149987 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.149889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35329 (local address 127.0.0.1:33206)
0923 22:12:06.149935 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.149961 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 22:12:06.150074 24647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:35329: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
W20260923 22:12:06.201073 24654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:06.201305 24654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:06.201323 24654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:06.201342 24654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:06.202638 24654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:06.202688 24654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:06.202703 24654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:06.202739 24654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:06.202768 24654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:06.204425 24654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:45739,127.1.244.253:35329,127.1.244.252:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:35329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.244.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:06.204706 24654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:06.204953 24654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:06.210012 24659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:06.210016 24662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:06.210012 24660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:06.210309 24654 server_base.cc:1034] running on GCE node
I20260923 22:12:06.210574 24654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:06.210846 24654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:06.211985 24654 hybrid_clock.cc:648] HybridClock initialized: now 1790201526211967 us; error 42 us; skew 500 ppm
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:06.214529 24654 init.cc:377] Logged in from keytab as kudu/127.1.244.253@KRBTEST.COM (short username kudu)
I20260923 22:12:06.215029 24654 webserver.cc:460] Webserver started at http://127.1.244.253:44705/ using document root <none> and password file <none>
I20260923 22:12:06.215274 24654 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:06.215320 24654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:06.215488 24654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:06.216617 24654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/data/instance:
uuid: "964aac56a59e44ecb8d925fb4bf1ddac"
format_stamp: "Formatted at 2026-09-23 22:12:06 on dist-test-slave-b5jz"
server_key: "02c577a57c7d3546d8fa408e72ebc9da"
server_key_iv: "0fe697262fbc1b8f80827330457a3180"
server_key_version: "encryptionkey@0"
I20260923 22:12:06.216991 24654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/wal/instance:
uuid: "964aac56a59e44ecb8d925fb4bf1ddac"
format_stamp: "Formatted at 2026-09-23 22:12:06 on dist-test-slave-b5jz"
server_key: "02c577a57c7d3546d8fa408e72ebc9da"
server_key_iv: "0fe697262fbc1b8f80827330457a3180"
server_key_version: "encryptionkey@0"
I20260923 22:12:06.218230 24647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } attempt: 2
I20260923 22:12:06.218457 24654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:06.218719 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.218512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35329 (local address 127.0.0.1:33216)
0923 22:12:06.218601 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.218677 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 22:12:06.219107 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.218992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35329 (local address 127.0.0.1:33230)
0923 22:12:06.219045 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.219078 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:12:06.219225 24647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:35329: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35329: connect: Connection refused (error 111)
I20260923 22:12:06.219514 24669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:06.219686 24654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:06.219789 24654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/wal
uuid: "964aac56a59e44ecb8d925fb4bf1ddac"
format_stamp: "Formatted at 2026-09-23 22:12:06 on dist-test-slave-b5jz"
server_key: "02c577a57c7d3546d8fa408e72ebc9da"
server_key_iv: "0fe697262fbc1b8f80827330457a3180"
server_key_version: "encryptionkey@0"
I20260923 22:12:06.219877 24654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:06.236111 24654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:06.238612 24654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:06.238795 24654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:06.242913 24654 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:35329
I20260923 22:12:06.242954 24721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:35329 every 8 connection(s)
I20260923 22:12:06.243350 24654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/data/info.pb
I20260923 22:12:06.244122 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 24654
I20260923 22:12:06.244266 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-1/wal/instance
I20260923 22:12:06.244490 2003 external_mini_cluster.cc:1428] Setting key 28ef5d8f56571f6cf2d06aa458c1e3f0
I20260923 22:12:06.244679 24722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:06.247108 24722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.244.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.252@KRBTEST.COM" created.
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:12:06.261075 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.248009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60040 (local address 127.1.244.254:45739)
0923 22:12:06.248085 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.248093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.248331 (+ 238us) server_negotiation.cc:407] Connection header received
0923 22:12:06.248937 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.248938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.248957 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.249014 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.250042 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.250536 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.252282 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.252500 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.252568 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.257591 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.257593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.257594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.257603 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.259466 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.259962 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.259964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.259964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.260038 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.260301 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.260302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.260302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.260479 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.260550 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.260837 (+ 287us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.260947 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 22:12:06.261163 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.247986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45739 (local address 127.0.0.1:60040)
0923 22:12:06.248194 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.248236 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:06.248806 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:06.249094 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:06.249099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:06.249544 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:06.249917 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.249926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.250642 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.250643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.252182 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.252186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.252623 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.252625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.252819 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.253425 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:06.253437 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:06.257446 (+ 4009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:06.259607 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.259621 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.259623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.259881 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.260113 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.260114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.260114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.260222 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.260543 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:06.260548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:06.260683 (+ 135us) client_negotiation.cc:769] Sending connection context
0923 22:12:06.260764 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:12:06.260901 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260923 22:12:06.261834 24722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260923 22:12:06.272706 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.262143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35329 (local address 127.0.0.1:33236)
0923 22:12:06.262200 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.262212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:06.262259 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:06.262653 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:06.262654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:06.262775 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:06.262832 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.262833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.264400 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.264400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.264911 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.264913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.265672 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.265672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.265725 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.266215 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:06.266219 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:06.269475 (+ 3256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:06.271671 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.271672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.271672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.271839 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.272046 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.272047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.272047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.272100 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.272413 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:06.272414 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:06.272455 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 22:12:06.272508 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:12:06.272625 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 22:12:06.272888 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.262228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.1.244.253:35329)
0923 22:12:06.262448 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.262454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.262486 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:12:06.262513 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.262515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.262533 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.262654 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.262913 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.264305 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.265001 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.265585 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.265680 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.269621 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.269631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.269631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.269639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.271500 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.271917 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.271921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.271921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.271981 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.272176 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.272176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.272176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.272356 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.272422 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.272555 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.272720 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":181,"thread_start_us":140,"threads_started":1}
I20260923 22:12:06.273541 24722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:06.274026 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.273860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50590)
0923 22:12:06.273929 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.273997 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
W20260923 22:12:06.274191 24671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
I20260923 22:12:06.274703 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.274595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50606)
0923 22:12:06.274647 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.274674 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 22:12:06.274855 24722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:37651: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
I20260923 22:12:06.275108 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6wdnYN/build/release/bin/kudu
/tmp/dist-test-task6wdnYN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:37651
--webserver_interface=127.1.244.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:45739,127.1.244.253:35329,127.1.244.252:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:06.310405 24647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } attempt: 3
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260923 22:12:06.318054 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.310834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35329 (local address 127.0.0.1:33246)
0923 22:12:06.310895 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.310915 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:06.310988 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:06.311142 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:06.311143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:06.311327 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:06.311395 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.311396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.312143 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.312145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.312701 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.312702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.313112 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.313113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.313202 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.313707 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:06.313713 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:06.315153 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:06.316938 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.316939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.316940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.317178 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.317380 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.317380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.317381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.317447 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.317699 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:06.317700 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:06.317758 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 22:12:06.317845 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 22:12:06.317951 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:12:06.318054 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.310921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33246 (local address 127.1.244.253:35329)
0923 22:12:06.311000 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.311002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.311015 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:12:06.311048 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.311050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.311061 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.311135 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.311487 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.312053 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.312794 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.313033 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.313115 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.315261 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.315262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.315263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.315271 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.316829 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.317239 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.317241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.317241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.317304 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.317510 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.317511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.317511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.317629 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.317707 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.317869 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.317952 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 22:12:06.318396 24647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:06.318826 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.318690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50616)
0923 22:12:06.318756 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.318796 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:12:06.319257 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.319159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50630)
0923 22:12:06.319204 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.319238 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 22:12:06.319363 24647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:37651: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
I20260923 22:12:06.324060 24722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } attempt: 1
I20260923 22:12:06.324447 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.324326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50644)
0923 22:12:06.324382 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.324411 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:12:06.324805 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.324689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50652)
0923 22:12:06.324740 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.324776 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:12:06.324908 24722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:37651: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
I20260923 22:12:06.350524 24647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } attempt: 1
I20260923 22:12:06.351248 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.351062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50656)
0923 22:12:06.351143 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.351190 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260923 22:12:06.351672 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.351577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50668)
0923 22:12:06.351615 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.351647 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 22:12:06.351805 24647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:37651: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
W20260923 22:12:06.358875 24729 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:06.359104 24729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:06.359122 24729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:06.359175 24729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:06.360424 24729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:06.360471 24729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:06.360486 24729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:06.360522 24729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:06.360546 24729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:06.362151 24729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:45739,127.1.244.253:35329,127.1.244.252:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.244.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:06.362433 24729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:06.362671 24729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:06.367503 24737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:06.367497 24734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:06.367497 24735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:06.367720 24729 server_base.cc:1034] running on GCE node
I20260923 22:12:06.368026 24729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:06.368295 24729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:06.369452 24729 hybrid_clock.cc:648] HybridClock initialized: now 1790201526369436 us; error 29 us; skew 500 ppm
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:06.372081 24729 init.cc:377] Logged in from keytab as kudu/127.1.244.252@KRBTEST.COM (short username kudu)
I20260923 22:12:06.372539 24729 webserver.cc:460] Webserver started at http://127.1.244.252:43725/ using document root <none> and password file <none>
I20260923 22:12:06.372783 24729 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:06.372835 24729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:06.372979 24729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:06.374050 24729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/data/instance:
uuid: "9ff8f30430234a51a992429aa09c37d0"
format_stamp: "Formatted at 2026-09-23 22:12:06 on dist-test-slave-b5jz"
server_key: "206f11eae65e757e4d63ec5a7fc9454f"
server_key_iv: "5d0dadd20714f8db43ef2a2608d18b18"
server_key_version: "encryptionkey@0"
I20260923 22:12:06.374414 24729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/wal/instance:
uuid: "9ff8f30430234a51a992429aa09c37d0"
format_stamp: "Formatted at 2026-09-23 22:12:06 on dist-test-slave-b5jz"
server_key: "206f11eae65e757e4d63ec5a7fc9454f"
server_key_iv: "5d0dadd20714f8db43ef2a2608d18b18"
server_key_version: "encryptionkey@0"
I20260923 22:12:06.375749 24729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:06.376495 24744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:06.376657 24729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:06.376765 24729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/data,/tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/wal
uuid: "9ff8f30430234a51a992429aa09c37d0"
format_stamp: "Formatted at 2026-09-23 22:12:06 on dist-test-slave-b5jz"
server_key: "206f11eae65e757e4d63ec5a7fc9454f"
server_key_iv: "5d0dadd20714f8db43ef2a2608d18b18"
server_key_version: "encryptionkey@0"
I20260923 22:12:06.376853 24729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:06.393044 24722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } attempt: 2
I20260923 22:12:06.393502 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.393374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50680)
0923 22:12:06.393435 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.393473 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:12:06.393882 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.393785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50686)
0923 22:12:06.393833 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.393857 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:12:06.393962 24722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:37651: Network error: Client connection negotiation failed: client connection to 127.1.244.252:37651: connect: Connection refused (error 111)
I20260923 22:12:06.404142 24729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:06.406157 24729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:06.406337 24729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:06.410426 24729 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:37651
I20260923 22:12:06.410471 24796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:37651 every 8 connection(s)
I20260923 22:12:06.410902 24729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/data/info.pb
I20260923 22:12:06.412014 24797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:06.413563 24797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:12:06.420217 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6wdnYN/build/release/bin/kudu as pid 24729
I20260923 22:12:06.420339 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6wdnYN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201256447402-2003-0/minicluster-data/master-2/wal/instance
I20260923 22:12:06.420562 2003 external_mini_cluster.cc:1428] Setting key 0a453bc0cc745f546749c67055e36f65
I20260923 22:12:06.423168 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.414301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.1.244.254:45739)
0923 22:12:06.414391 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.414394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.414589 (+ 195us) server_negotiation.cc:407] Connection header received
0923 22:12:06.415040 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.415041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.415051 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.415106 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.415803 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.416271 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.417241 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.417448 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.417507 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.419754 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.419756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.419756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.419765 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.421553 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.422069 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.422070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.422071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.422135 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.422384 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.422386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.422386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.422523 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.422592 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.422968 (+ 376us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.423047 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 22:12:06.423326 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.414263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45739 (local address 127.0.0.1:60042)
0923 22:12:06.414467 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.414504 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:06.414955 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:06.415171 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:06.415174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:06.415445 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:06.415724 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.415729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.416359 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.416360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.417153 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.417155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.417538 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.417539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.417638 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.418174 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:06.418191 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:06.419669 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:06.421628 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.421635 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.421638 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.421992 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.422210 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.422211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.422211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.422315 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.422629 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:06.422634 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:06.422804 (+ 170us) client_negotiation.cc:769] Sending connection context
0923 22:12:06.422922 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 22:12:06.423085 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":117,"threads_started":1}
I20260923 22:12:06.423897 24797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260923 22:12:06.426985 24647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } attempt: 2
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260923 22:12:06.430609 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.420941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50690 (local address 127.1.244.252:37651)
0923 22:12:06.421159 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.421164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.421218 (+ 54us) server_negotiation.cc:407] Connection header received
0923 22:12:06.421273 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.421275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.421290 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.421381 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.422109 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.423082 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.424064 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.424504 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.424577 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.427118 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.427127 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.427127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.427134 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.429197 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.429726 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.429729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.429729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.429793 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.429999 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.430000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.430001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.430193 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.430365 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.430413 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.430492 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":111,"threads_started":1}
I20260923 22:12:06.431533 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.424252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.1.244.253:35329)
0923 22:12:06.424312 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.424314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.424341 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:12:06.424411 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.424411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.424420 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.424463 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.424761 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.425470 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.426232 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.426429 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.426479 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.428458 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.428459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.428460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.428467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.430060 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.430424 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.430425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.430426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.430484 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.430949 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.430950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.430950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.431088 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.431155 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.431355 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.431437 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:12:06.431974 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.424186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35329 (local address 127.0.0.1:33258)
0923 22:12:06.424297 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.424308 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:06.424354 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:06.424466 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:06.424466 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:06.424628 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:06.424682 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.424684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.425558 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.425559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.426125 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.426128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.426498 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.426499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.426570 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.426972 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:06.426977 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:06.428371 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:06.430128 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.430129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.430129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.430329 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.430563 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.430564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.430564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.430886 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.431194 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:06.431195 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:06.431235 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 22:12:06.431777 (+ 542us) client_negotiation.cc:240] Negotiation successful
0923 22:12:06.431881 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260923 22:12:06.432475 24797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260923 22:12:06.436376 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.427230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50692)
0923 22:12:06.427284 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.427296 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:06.427342 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:06.427632 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:06.427634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:06.427779 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:06.427826 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.427828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.429059 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.429060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.429875 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.429878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.430423 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.430424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.430484 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.430866 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:06.430871 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:06.432475 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:06.434933 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.434934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.434935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.435175 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.435389 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.435390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.435391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.435462 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.436091 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:06.436092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:06.436140 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 22:12:06.436193 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:12:06.436278 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:12:06.436818 24802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.427277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.1.244.252:37651)
0923 22:12:06.427433 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.427435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.427443 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:06.427463 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.427468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.427476 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.427619 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.427916 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.428966 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.429977 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.430323 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.431098 (+ 775us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.432589 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.432590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.432590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.432597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.434843 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.435260 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.435260 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.435261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.435320 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.435541 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.435566 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.435567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.435704 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.436578 (+ 874us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.436624 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.436695 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260923 22:12:06.438869 24647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725: Bootstrap starting.
I20260923 22:12:06.439733 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.431448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.1.244.254:45739)
0923 22:12:06.431490 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.431492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.431500 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:06.431531 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.431532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.431540 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.431610 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.432298 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.433001 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.433843 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.434119 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.434179 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.436719 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.436721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.436721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.436727 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.438314 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.438689 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.438690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.438691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.438749 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.439036 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.439037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.439037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.439159 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.439277 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.439539 (+ 262us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.439648 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:06.439980 24647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:06.440187 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.432737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50706)
0923 22:12:06.432777 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.432787 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:06.432821 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:06.433220 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:06.433221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:06.433372 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:06.433403 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.433404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.434314 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.434315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.435062 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.435064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.435564 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.435565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.435614 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.436030 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:06.436035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:06.437407 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:06.439212 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.439213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.439214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.439380 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.439610 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.439611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.439611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.439665 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.439940 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:06.439941 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:06.439979 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:12:06.440041 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:12:06.440131 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260923 22:12:06.440311 24647 log.cc:824] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:06.440470 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.432765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.1.244.252:37651)
0923 22:12:06.433104 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.433106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.433113 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:06.433130 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.433131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.433141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.433195 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.433482 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.434227 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.435136 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.435464 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.435572 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.437511 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.437512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.437512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.437519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.439109 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.439452 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.439453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.439453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.439503 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.439736 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.439737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.439737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.439871 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.439948 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.440306 (+ 358us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.440385 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319}
I20260923 22:12:06.441520 24647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725: No bootstrap required, opened a new log
I20260923 22:12:06.441686 24797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0: Bootstrap starting.
I20260923 22:12:06.442273 24797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:06.442553 24797 log.cc:824] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:06.443284 24797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0: No bootstrap required, opened a new log
I20260923 22:12:06.443639 24647 raft_consensus.cc:348] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } }
I20260923 22:12:06.443800 24647 raft_consensus.cc:729] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35eaf617af048de975333061be8f725, State: Initialized, Role: FOLLOWER
I20260923 22:12:06.443902 24647 consensus_queue.cc:260] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } }
I20260923 22:12:06.444186 24805 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.444269 24647 sys_catalog.cc:564] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:06.445286 24797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } }
I20260923 22:12:06.445456 24797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff8f30430234a51a992429aa09c37d0, State: Initialized, Role: FOLLOWER
I20260923 22:12:06.444314 24805 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [sys.catalog]: This master's current role is: FOLLOWER
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
W20260923 22:12:06.446239 24816 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:06.446285 24816 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:06.445545 24797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } }
I20260923 22:12:06.447140 24797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [sys.catalog]: configured and running, proceeding with master startup.
W20260923 22:12:06.449035 24829 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:06.449086 24829 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:06.450290 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.440984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33268 (local address 127.1.244.253:35329)
0923 22:12:06.441028 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.441030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.441039 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:06.441066 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.441067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.441076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.441134 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.441733 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.442424 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.443400 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.443733 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.443781 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.446837 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.446838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.446839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.446847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.448734 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.449161 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.449162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.449162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.449230 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.449444 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.449445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.449445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.449591 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.449740 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.450122 (+ 382us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.450198 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:06.448150 24579 master_runner.cc:186] Error getting master registration for 127.1.244.253:35329: OK, instance_id { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" instance_seqno: 1790201526241796 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:12:06.451047 24579 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:12:06.452909 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:06.453810 24621 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d35eaf617af048de975333061be8f725"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60052
I20260923 22:12:06.453923 24621 raft_consensus.cc:482] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 22:12:06.453969 24621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:06.454890 24621 raft_consensus.cc:504] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } }
I20260923 22:12:06.455442 24621 leader_election.cc:288] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [CANDIDATE]: Term 1 election: Requested vote from peers 964aac56a59e44ecb8d925fb4bf1ddac (127.1.244.253:35329), 9ff8f30430234a51a992429aa09c37d0 (127.1.244.252:37651)
I20260923 22:12:06.455941 24771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d35eaf617af048de975333061be8f725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9ff8f30430234a51a992429aa09c37d0"
I20260923 22:12:06.456044 24771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:06.456923 24771 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35eaf617af048de975333061be8f725 in term 1.
I20260923 22:12:06.457124 24596 leader_election.cc:302] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ff8f30430234a51a992429aa09c37d0, d35eaf617af048de975333061be8f725; no voters:
I20260923 22:12:06.457757 24818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.457829 24818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:06.458402 24805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:06.458604 24805 raft_consensus.cc:686] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [term 1 LEADER]: Becoming Leader. State: Replica: d35eaf617af048de975333061be8f725, State: Running, Role: LEADER
I20260923 22:12:06.458756 24805 consensus_queue.cc:237] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } }
I20260923 22:12:06.459281 24805 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d35eaf617af048de975333061be8f725. Latest consensus state: current_term: 1 leader_uuid: "d35eaf617af048de975333061be8f725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.459424 24805 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:06.459787 24837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:06.457690 24729 master_runner.cc:186] Error getting master registration for 127.1.244.253:35329: OK, instance_id { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" instance_seqno: 1790201526241796 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:12:06.460469 24837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:06.460536 24729 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:12:06.462301 24771 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [term 1 FOLLOWER]: Refusing update from remote peer d35eaf617af048de975333061be8f725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:06.462812 24841 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:06.464254 24818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d35eaf617af048de975333061be8f725. Latest consensus state: current_term: 1 leader_uuid: "d35eaf617af048de975333061be8f725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.464337 24818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:06.465600 24841 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d35eaf617af048de975333061be8f725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.465704 24841 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:06.465763 24838 mvcc.cc:205] Tried to move back new op lower bound from 7332665452386988032 to 7332665452376244224. Current Snapshot: MvccSnapshot[applied={T|T < 7332665452386988032}]
I20260923 22:12:06.466204 24818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d35eaf617af048de975333061be8f725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.466293 24818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:06.466564 24842 mvcc.cc:205] Tried to move back new op lower bound from 7332665452386988032 to 7332665452376244224. Current Snapshot: MvccSnapshot[applied={T|T < 7332665452386988032}]
I20260923 22:12:06.471475 24837 catalog_manager.cc:1347] Generated new cluster ID: c7e0783ee851442aae8a192bc8d74da8
I20260923 22:12:06.471531 24837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:06.476089 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.461477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.1.244.252:37651)
0923 22:12:06.461530 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.461532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.464941 (+ 3409us) server_negotiation.cc:407] Connection header received
0923 22:12:06.465112 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.465126 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.465137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.465167 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.466572 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.467338 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.469280 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.469604 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.469646 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.472377 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.472378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.472378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.472386 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.474270 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.474960 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.474962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.474962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.475018 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.475329 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.475330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.475330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.475477 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.475578 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.475907 (+ 329us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.475992 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:06.485085 24722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } attempt: 3
I20260923 22:12:06.486503 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.461882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.1.244.254:45739)
0923 22:12:06.461924 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.461926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.466917 (+ 4991us) server_negotiation.cc:407] Connection header received
0923 22:12:06.467028 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.467029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.467039 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.467069 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.468916 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.469642 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.479671 (+ 10029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.480034 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.480080 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.482947 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.482949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.482949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.482957 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.484655 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.485362 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.485363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.485363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.485427 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.485764 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.485765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.485765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.485911 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.485993 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.486322 (+ 329us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.486408 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 23 22:12:06 dist-test-slave-b5jz krb5kdc[24566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260923 22:12:06.494107 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.485410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:37651 (local address 127.0.0.1:50724)
0923 22:12:06.485551 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:06.485563 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:06.485602 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:06.485870 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:06.485871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:06.486040 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:06.486088 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.486089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.487290 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.487292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.488104 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:06.488106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.488986 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.488987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:06.489048 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.489585 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:06.489590 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:06.491187 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:06.493031 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.493033 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.493033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.493245 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.493443 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:06.493443 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:06.493444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:06.493500 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:06.493801 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:06.493802 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:06.493847 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 22:12:06.493909 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:12:06.494004 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260923 22:12:06.494627 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.485692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.1.244.252:37651)
0923 22:12:06.485734 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.485736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.485744 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:06.485767 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.485767 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.485777 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.485823 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.486164 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.486907 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.488205 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.488501 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.488556 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.491285 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.491287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.491287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.491295 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.492938 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.493323 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.493324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.493324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.493379 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.493564 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.493565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.493565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.493714 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.493775 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.494458 (+ 683us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.494537 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:06.496575 24722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac: Bootstrap starting.
I20260923 22:12:06.497320 24722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:06.497617 24722 log.cc:824] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:06.500888 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:06.464600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.1.244.253:35329)
0923 22:12:06.464659 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.464661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.467984 (+ 3323us) server_negotiation.cc:407] Connection header received
0923 22:12:06.468092 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.468094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.468105 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.468136 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.479643 (+ 11507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.480438 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.481598 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:06.481927 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:06.481969 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:06.497890 (+ 15921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:06.497891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:06.497892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:06.497899 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:06.499661 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.500090 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.500091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.500091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.500141 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:06.500330 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:06.500331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:06.500331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:06.500455 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:06.500531 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:06.500739 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 22:12:06.500811 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:06.502360 24722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac: No bootstrap required, opened a new log
I20260923 22:12:06.504444 24722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } }
I20260923 22:12:06.504585 24722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964aac56a59e44ecb8d925fb4bf1ddac, State: Initialized, Role: FOLLOWER
I20260923 22:12:06.504666 24722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } }
I20260923 22:12:06.505040 24722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:06.505127 24696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d35eaf617af048de975333061be8f725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "964aac56a59e44ecb8d925fb4bf1ddac"
I20260923 22:12:06.505254 24696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:06.506140 24696 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35eaf617af048de975333061be8f725 in term 1.
I20260923 22:12:06.506354 24695 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [term 1 FOLLOWER]: Refusing update from remote peer d35eaf617af048de975333061be8f725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:06.507625 24835 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:06.509274 24862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d35eaf617af048de975333061be8f725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.509372 24862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:06.509423 24862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [sys.catalog]: SysCatalogTable state changed. Reason: New leader d35eaf617af048de975333061be8f725. Latest consensus state: current_term: 1 leader_uuid: "d35eaf617af048de975333061be8f725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.509462 24862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:06.509488 24862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d35eaf617af048de975333061be8f725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.509523 24862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:12:06.511181 24863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:06.511245 24863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:06.511965 24860 mvcc.cc:205] Tried to move back new op lower bound from 7332665452386988032 to 7332665452376244224. Current Snapshot: MvccSnapshot[applied={T|T < 7332665452386988032}]
I20260923 22:12:06.512166 24837 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:06.512960 24837 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:06.519814 24841 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d35eaf617af048de975333061be8f725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35eaf617af048de975333061be8f725" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45739 } } peers { permanent_uuid: "964aac56a59e44ecb8d925fb4bf1ddac" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35329 } } peers { permanent_uuid: "9ff8f30430234a51a992429aa09c37d0" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 37651 } } }
I20260923 22:12:06.519922 24841 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:06.521658 24837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d35eaf617af048de975333061be8f725: Generated new TSK 0
I20260923 22:12:06.521880 24837 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:12:06.543623 24801 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.542011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50738 (local address 127.1.244.252:37651)
0923 22:12:06.542063 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.542065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.542941 (+ 876us) server_negotiation.cc:407] Connection header received
0923 22:12:06.543047 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.543048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.543059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.543090 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.543512 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50738: BlockingRecv error: recv got EOF from 127.0.0.1:50738 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260923 22:12:06.544538 24650 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.542381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.1.244.254:45739)
0923 22:12:06.542419 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.542421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.544044 (+ 1623us) server_negotiation.cc:407] Connection header received
0923 22:12:06.544141 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.544142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.544153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.544182 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.544481 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60066: BlockingRecv error: recv got EOF from 127.0.0.1:60066 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:12:06.545377 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.542651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33296 (local address 127.1.244.253:35329)
0923 22:12:06.542730 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.542731 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.544888 (+ 2157us) server_negotiation.cc:407] Connection header received
0923 22:12:06.544979 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.544980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.544991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.545020 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.545321 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33296: BlockingRecv error: recv got EOF from 127.0.0.1:33296 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:12:06.547802 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.547053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33302 (local address 127.1.244.253:35329)
0923 22:12:06.547089 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.547090 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.547265 (+ 175us) server_negotiation.cc:407] Connection header received
0923 22:12:06.547369 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.547370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.547388 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.547417 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.547757 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33302: BlockingRecv error: recv got EOF from 127.0.0.1:33302 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:12:06.550844 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.550128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.1.244.253:35329)
0923 22:12:06.550161 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.550163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.550370 (+ 207us) server_negotiation.cc:407] Connection header received
0923 22:12:06.550456 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.550456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.550466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.550494 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.550803 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33304: BlockingRecv error: recv got EOF from 127.0.0.1:33304 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:12:06.554899 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.554146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33320 (local address 127.1.244.253:35329)
0923 22:12:06.554181 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.554183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.554392 (+ 209us) server_negotiation.cc:407] Connection header received
0923 22:12:06.554490 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.554491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.554501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.554530 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.554856 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33320: BlockingRecv error: recv got EOF from 127.0.0.1:33320 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:12:06.561298 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.560464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33330 (local address 127.1.244.253:35329)
0923 22:12:06.560503 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.560505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.560740 (+ 235us) server_negotiation.cc:407] Connection header received
0923 22:12:06.560845 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.560846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.560857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.560888 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.561247 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33330: BlockingRecv error: recv got EOF from 127.0.0.1:33330 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:12:06.571722 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.570771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33336 (local address 127.1.244.253:35329)
0923 22:12:06.570816 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.570819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.571076 (+ 257us) server_negotiation.cc:407] Connection header received
0923 22:12:06.571184 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.571185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.571197 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.571229 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.571654 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33336: BlockingRecv error: recv got EOF from 127.0.0.1:33336 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 22:12:06.590219 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.589285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.1.244.253:35329)
0923 22:12:06.589333 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.589336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.589592 (+ 256us) server_negotiation.cc:407] Connection header received
0923 22:12:06.589716 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.589718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.589731 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.589762 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.590156 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33344: BlockingRecv error: recv got EOF from 127.0.0.1:33344 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 22:12:06.625849 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.623882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33348 (local address 127.1.244.253:35329)
0923 22:12:06.623927 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.623930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.625197 (+ 1267us) server_negotiation.cc:407] Connection header received
0923 22:12:06.625313 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.625315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.625328 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.625359 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.625777 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33348: BlockingRecv error: recv got EOF from 127.0.0.1:33348 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:12:06.692112 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.691391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33358 (local address 127.1.244.253:35329)
0923 22:12:06.691476 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.691479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.691515 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:12:06.691613 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.691614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.691632 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.691684 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.692051 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33358: BlockingRecv error: recv got EOF from 127.0.0.1:33358 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260923 22:12:06.822126 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:06.821281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.1.244.253:35329)
0923 22:12:06.821357 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:06.821360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:06.821461 (+ 101us) server_negotiation.cc:407] Connection header received
0923 22:12:06.821565 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:06.821566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:06.821585 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:06.821653 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:06.822056 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33364: BlockingRecv error: recv got EOF from 127.0.0.1:33364 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260923 22:12:07.080085 24727 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:07.079322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.1.244.253:35329)
0923 22:12:07.079438 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:07.079440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:07.079455 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:12:07.079510 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:07.079511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:07.079522 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:07.079597 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:07.080022 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33374: BlockingRecv error: recv got EOF from 127.0.0.1:33374 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 22:12:07.449506 24829 catalog_manager.cc:1259] Loaded cluster ID: c7e0783ee851442aae8a192bc8d74da8
I20260923 22:12:07.449604 24829 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0: loading cluster ID for follower catalog manager: success
I20260923 22:12:07.450419 24829 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0: acquiring CA information for follower catalog manager: success
I20260923 22:12:07.450730 24829 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9ff8f30430234a51a992429aa09c37d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:12:07.511950 24863 catalog_manager.cc:1259] Loaded cluster ID: c7e0783ee851442aae8a192bc8d74da8
I20260923 22:12:07.512039 24863 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac: loading cluster ID for follower catalog manager: success
I20260923 22:12:07.513001 24863 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac: acquiring CA information for follower catalog manager: success
I20260923 22:12:07.513377 24863 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 964aac56a59e44ecb8d925fb4bf1ddac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:12:07.595880 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:07.593328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.1.244.253:35329)
0923 22:12:07.593438 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:07.593440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:07.593449 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:07.593477 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:07.593478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:07.593531 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:07.593622 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:07.593825 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.594333 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.595091 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.595283 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.595340 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:07.595484 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:07.595616 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:07.595676 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:07.595720 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 22:12:07.595738 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260923 22:12:07.600451 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:07.597817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50740 (local address 127.1.244.252:37651)
0923 22:12:07.597910 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:07.597912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:07.597950 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:12:07.598029 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:07.598029 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:07.598064 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:07.598137 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:07.598257 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.598754 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.599486 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.599800 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.599874 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:07.600043 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:07.600171 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:07.600257 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:07.600302 (+ 45us) server_negotiation.cc:299] Negotiation successful
0923 22:12:07.600324 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":49,"threads_started":1}
I20260923 22:12:07.605369 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:07.602412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33400 (local address 127.1.244.253:35329)
0923 22:12:07.602455 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:07.602457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:07.602535 (+ 78us) server_negotiation.cc:407] Connection header received
0923 22:12:07.602600 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:07.602602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:07.602613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:07.602669 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:07.602820 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.603565 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.604356 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.604661 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.604732 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:07.604938 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:07.604975 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:07.605032 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:07.605280 (+ 248us) server_negotiation.cc:299] Negotiation successful
0923 22:12:07.605308 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:07.605362 24943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:07.602113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.1.244.254:45739)
0923 22:12:07.602283 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:07.602285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:07.602295 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:07.602378 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:07.602379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:07.602433 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:07.602700 (+ 267us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:07.602746 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.603266 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.604383 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.604659 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.604739 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:07.604931 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:07.605059 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:07.605121 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:07.605164 (+ 43us) server_negotiation.cc:299] Negotiation successful
0923 22:12:07.605190 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":47,"threads_started":1}
I20260923 22:12:07.605477 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:07.602205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.1.244.252:37651)
0923 22:12:07.602250 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:07.602252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:07.602402 (+ 150us) server_negotiation.cc:407] Connection header received
0923 22:12:07.602501 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:07.602502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:07.602513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:07.602561 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:07.602761 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.603496 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.604518 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:07.604811 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:07.604885 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:07.605112 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:07.605144 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:07.605175 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:07.605351 (+ 176us) server_negotiation.cc:299] Negotiation successful
0923 22:12:07.605374 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:07.612847 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 24579
I20260923 22:12:07.618446 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 24654
I20260923 22:12:07.623278 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6wdnYN/build/release/bin/kudu with pid 24729
2026-09-23T22:12:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2826 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5679 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (271183 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST